A dashboard reading identification method, device, equipment and medium

Abstract

The application provides a kind of instrument panel reading identification method, device, equipment and medium, it is related to data processing technical field, including: the key point detection is carried out to the instrument panel image obtained and is preprocessed, to obtain the image to be handled containing instrument panel scale area and pointer;In the image to be handled, each scale interval in scale area is positioned by key point, and the scale area is transformed and corrected based on each key point, so that the image corresponding to the scale area is converted into rectangular image;Based on the key points at both ends of the scale area, image processing is carried out to generate pointer rotation information, and the mapping data corresponding to the rotation of the pointer is determined according to the angle mapping of the pointer rotation information;According to the mapping data corresponding to the rotation of the pointer and the rectangular image corresponding to the scale area, the position of the pointer under the corresponding scale interval of the rectangular image is determined, to obtain the target reading corresponding to the pointer, to solve the problem that the existing method for reading instrument panel reading is complex in operation and the application scene is limited.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and medium for identifying dashboard readings. Background Technology

[0002] With the rapid development of technology, the scale and complexity of power systems have increased dramatically. Power maintenance, as one of the core operations of power systems, relies heavily on real-time data from various instruments. Collecting this data requires significant manpower and time. Current methods for reading pointer instrument panel data primarily depend on manual labor. In special scenarios, such as high-voltage substations and high-altitude equipment, manual inspections are laborious, the operating environment is complex, and there are significant health risks to operators. Existing computer vision-based methods for reading pointer instrument panel data are challenging due to the complexity of the data processing. These methods typically involve setting thresholds for binarization of the instrument panel image, segmenting the image after processing to extract key information such as scales and pointers, and then performing template matching to read the data. However, these methods have limited applicability to various equipment types and scenarios, and their accuracy is not high enough. Summary of the Invention

[0003] In order to overcome the above-mentioned technical defects, the purpose of this invention is to provide a method, device, equipment and medium for identifying dashboard readings, and to solve the problems that existing methods for reading dashboard readings are complicated to operate and have limited application scenarios.

[0004] This invention discloses a method for recognizing dashboard readings, comprising:

[0005] After key point detection, the acquired dashboard image is transformed and corrected for the first time to obtain a processed image containing the dashboard scale area and pointers.

[0006] In the image to be processed, each scale interval in the scale region is located by key points, and a second transformation correction is performed on the scale region based on each key point to convert the image corresponding to the scale region into a rectangular image.

[0007] Image processing is performed based on the preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information. Angle mapping is then performed based on the pointer rotation information to determine the corresponding mapping data for pointer rotation.

[0008] The position of the pointer under the corresponding scale interval of the rectangular image is determined based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, so as to obtain the target reading corresponding to the pointer.

[0009] Preferably, in the image to be processed, each scale interval in the scale region is located by key points, and a second transformation correction is performed on the scale region based on each key point to convert the arc image corresponding to the scale region into a rectangular image, including: sequentially marking each scale interval in the scale region, wherein each scale interval includes two endpoints, and each endpoint corresponds to two ends;

[0010] Four key points are used to locate the four ends of each scale interval to obtain quadrilateral sub-blocks corresponding to each scale interval, which are composed of each key point.

[0011] After performing the second transformation correction on each quadrilateral sub-block, they are spliced ​​together according to the sequence markings, so that the arc-shaped image of the scale area is converted into a rectangular image.

[0012] Preferably, two key points are shared at the end of the splicing side of two adjacent scale intervals.

[0013] Preferably, the step of performing image processing based on a preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information, and determining the corresponding mapping data for pointer rotation based on the pointer rotation information, includes:

[0014] The minimum and maximum rotation angles of the pointer relative to the horizontal direction are determined based on the preset pointer rotation point and the key points at both ends of the scale area.

[0015] The actual rotation angle of the pointer relative to the horizontal direction is determined by image processing based on the preset pointer rotation point;

[0016] Calculate the first difference between the actual rotation angle and the minimum rotation angle, calculate the second difference between the maximum rotation angle and the minimum rotation angle, and determine the mapping data corresponding to the pointer rotation based on the ratio of the first difference and the second difference.

[0017] Preferably, determining the position of the pointer under the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area to obtain the target reading corresponding to the pointer includes:

[0018] Obtain the first and second endpoints of the corresponding scale interval under the scale area where the pointer is located;

[0019] Obtain the first scale value and the second scale value corresponding to the first endpoint and the second endpoint, respectively;

[0020] Obtain the total length data of the rectangular image corresponding to the scale area; determine the first position data based on the length of the first endpoint from the key point at the beginning of the scale area and the total length data; determine the second position data based on the length of the second endpoint from the key point at the beginning of the scale area and the total length data.

[0021] Based on the scale changes of the first and second scale values ​​relative to the position changes of the first and second position data, the relationship between scale and position changes within the scale interval where the pointer is located is determined.

[0022] Based on the relationship between the scale and the position change, and the change of the mapping data corresponding to the pointer rotation relative to the first position data / second position data, the scale change of the pointer relative to the first scale value / second scale value is determined, so as to obtain the target reading of the pointer.

[0023] Preferably, when the scale values ​​corresponding to each scale interval are evenly distributed, the step of determining the position of the pointer within the scale interval corresponding to the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, in order to obtain the target reading corresponding to the pointer, includes:

[0024] Obtain the scale values ​​at the beginning and end of the scale area to calculate the scale variation range;

[0025] The change in the pointer relative to the starting point of the scale area is determined by multiplying the scale change range with the mapping data, and the target reading corresponding to the pointer is obtained.

[0026] Preferably, the transformation correction involves performing one or more of the following operations on the acquired dashboard image: stretching, scaling, and cropping, after key point detection.

[0027] The present invention also provides an instrument panel reading recognition device, comprising:

[0028] The preprocessing module is used to perform key point detection and first transformation correction on the acquired dashboard image to obtain the image to be processed, which includes the dashboard scale area and pointers.

[0029] The scale region correction module is used to locate each scale interval in the scale region in the image to be processed by key points, and to perform a second transformation correction on the scale region based on each key point, so that the image corresponding to the scale region is converted into a rectangular image.

[0030] The pointer rotation recognition module is used to perform image processing based on a preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information, and to determine the corresponding mapping data of pointer rotation by performing angle mapping based on the pointer rotation information.

[0031] The calculation module is used to determine the position of the pointer under the corresponding scale interval of the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, so as to obtain the target reading corresponding to the pointer.

[0032] The present invention also provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the dashboard reading recognition method.

[0033] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the dashboard reading recognition method.

[0034] Compared with existing technologies, the above technical solution has the following advantages:

[0035] The dashboard reading recognition method provided in this embodiment transforms the scale area into a rectangular image by stretching it, and determines the movement position of the pointer on the rectangular image corresponding to the rotation of the pointer. It also determines the change of the pointer relative to the corresponding endpoint of the scale interval to calculate the target reading. This method has better accuracy in reading numerical values ​​of pointer-type dashboards, and is very robust to pointer-type dashboard images with tilt angles. It can realize data reading of pointer-type dashboards with arbitrary curvature and size, and has high accuracy. It is easy to operate and has wide applications. Attached Figure Description

[0036] Figure 1 This is a flowchart of a first embodiment of the dashboard reading recognition method described in this invention;

[0037] Figure 2 This is a flowchart illustrating how an arc-shaped image corresponding to a scale area is converted into a rectangular image in Embodiment 1 of the dashboard reading recognition method of the present invention.

[0038] Figure 3 This is a flowchart illustrating the process of determining the mapping data corresponding to pointer rotation in Embodiment 1 of the dashboard reading recognition method of the present invention.

[0039] Figure 4 This is a flowchart illustrating the process of obtaining the target reading corresponding to the pointer in an embodiment of the dashboard reading recognition method of the present invention;

[0040] Figure 5 This is a schematic diagram of a module in a second embodiment of the dashboard reading recognition device of the present invention;

[0041] Figure 6 This is a schematic diagram of the modules of the computer device described in this invention.

[0042] Figure label:

[0043] 5-Instrument panel reading recognition device; 51-Preprocessing module; 52-Scale area correction module; 53-Pointer rotation recognition module; 54-Calculation module; 6-Computer equipment; 61-Memory; 62-Processor. Detailed Implementation

[0044] The advantages of the present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments.

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0046] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0047] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0048] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0049] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0050] In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of the description of the invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0051] Example 1: This invention discloses a dashboard reading recognition method, proposing a novel method for reading pointer dashboard data. This method can efficiently and accurately read pointer data and can be applied to various scenarios. It involves acquiring dashboard images and then combining image analysis and data processing. For details, please refer to [link to relevant documentation]. Figure 1 This includes the following steps:

[0052] S100: After key point detection, the acquired dashboard image is transformed and corrected for the first time to obtain the image to be processed, which includes the dashboard scale area and pointers.

[0053] In the above steps, dashboard images (such as dashboard panel images) are acquired in real time by an image acquisition device, and key point detection is performed on the acquired dashboard images. In some embodiments, the image acquisition device includes a processing module that can directly perform key point detection on the dashboard images. Since different angles at which the image acquisition device acquires images may result in tilted dashboard panel images, i.e., images that are not parallel to the shooting plane, the above preprocessing mainly serves to extract the complete dashboard panel. Specifically, the transformation correction involves performing one or more operations such as stretching, scaling, and cropping on the acquired dashboard image after key point detection. It should be noted that the key point detection on the acquired dashboard image can be the key points of the entire dashboard panel, or it can be the key points containing only the ends of the scale area or the outline of the pointer. The key point detection can be achieved by a preset key point detection model (which can be a simple improved target detection model based on YOLOv5s, which is only an example and not a limitation. Those skilled in the art should understand that other existing or possible target detection models that meet the requirements of this application are applicable to this application). Based on the extracted key points, the input image (i.e., the dashboard image) is transformed and corrected (such as perspective transformation) to correct the tilted pointer-type dashboard image to a standard image with the panel parallel to the shooting plane, which facilitates the subsequent recognition of the pointer degree. This method can cope with many extreme scenarios and greatly reduces the difficulty in recognition caused by tilted shooting.

[0054] S200: In the image to be processed, each scale interval in the scale region is located by key points, and the scale region is transformed and corrected a second time based on each key point, so that the image corresponding to the scale region is converted into a rectangular image.

[0055] As can be seen from step S100 above, after the first transformation and correction, the image changes from a larger output image to a smaller image containing only the dashboard. In this embodiment, considering that the dial may be a curved distribution (potentially forming a fan-shaped or arc-shaped image), the scale area is regarded as curved text. Key point detection is performed on the curved text. A text key point detection model (or other detection model) can be set to achieve the above-mentioned key point localization. Specifically, S key points that can contain complete scale lines are extracted (the value of S is related to the number of scale intervals marked on the dashboard). The role of these key points is to describe the positional information of each scale line. In order to achieve the above-mentioned localization, a target detection model for scale lines can also be used. The main purpose is to detect the scale values ​​marked on the scale lines to facilitate key point localization and subsequent use.

[0056] Based on the key points used in the image to be processed, each scale interval in the scale region is located. A second transformation correction is then performed on the scale region based on each key point, converting the arc-shaped image corresponding to the scale region into a rectangular image. (See [reference]). Figure 2 ,include:

[0057] S210: Sequentially mark each scale interval in the scale region, wherein each scale interval includes two endpoints, and each endpoint corresponds to two ends;

[0058] Specifically, the scale area is the region where the pointer can rotate. Assuming the pointer can rotate 0-160°, the scale area is the region within 0-160°, containing several scale intervals, such as 10°-15°. These intervals can be evenly spaced or unevenly spaced; the identification method provided in this embodiment is applicable to both. Based on the scale area obtained above, it is grouped according to the order of the scale values, i.e., each scale interval within 0-160° includes two endpoints, and each endpoint corresponds to two ends.

[0059] S220: Four key points are used to locate the four ends in each scale interval to obtain a quadrilateral sub-block corresponding to each scale interval, which is composed of each key point.

[0060] In the above steps, each scale interval, such as 10°-15°, is formed between the scale line corresponding to 10° and the scale line corresponding to 15°. Each scale interval includes two endpoints, and each endpoint corresponds to two ends. Key point positioning is performed at the four ends in each scale interval, which are the four key points corresponding to that scale area. These four key points form the quadrilateral sub-blocks mentioned above. For example, if the scale area of ​​the instrument panel is 0°-160°, and each 5° is a scale interval, then the instrument panel has 32 quadrilateral sub-blocks.

[0061] S230: After performing the second transformation correction on each quadrilateral sub-block, they are spliced ​​together according to the sequence mark, so that the arc-shaped image of the scale area is converted into a rectangular image.

[0062] In the above steps, the quadrilateral sub-blocks obtained based on all keypoint sets are transformed and corrected. This results in the irregular shapes of the quadrilateral sub-blocks corresponding to each scale line in the curved scale region, as described above. After a second transformation and correction, these sub-blocks are transformed into regular rectangles and then stitched together according to the aforementioned scale value data order to form a rectangular image of the corresponding scale region. In other words, this can be viewed as stretching, scaling, and cropping the curved strips corresponding to the scale region to rectangular strips, which is achieved by stretching, scaling, and cropping the aforementioned quadrilateral sub-blocks to different degrees.

[0063] It should be noted that if each scale region corresponds to four key points, the number of key points is relatively large. When the various scale intervals are spliced ​​together, the key points at both ends of the scale lines shared by two adjacent scale intervals will overlap. Therefore, it is possible to make two key points shared at one end of the splicing side of two adjacent scale intervals, that is, S key points, of which each group has 4 key points. Excluding the two key points at the beginning end and the two key points at the end end of the scale region, the other key points belong to the two adjacent groups of key points respectively. The above can yield a total of S / 2-1 corrected quadrilateral sub-blocks.

[0064] S300: Based on the preset pointer rotation point and key points at both ends of the scale area, perform image processing to generate pointer rotation information, and determine the corresponding mapping data for pointer rotation by performing angle mapping based on the pointer rotation information;

[0065] In the above steps, image processing is performed based on the image to be processed (which may be combined with key points). Specifically, the image to be processed is input into a semantic segmentation model or object detection model to obtain the semantic segmentation result of the pointer. To determine the pointer's rotation position, a pointer rotation point is preset, representing the pointer's rotation position. The point around which the pointer rotates is recorded as point 0. Based on the preset pointer rotation point and two key points at both ends of the scale area (start and end), the rotation angle of the pointer when it rotates around point 0 and points to the scale is calculated (for example, clockwise rotation is assumed to be the direction of increasing angle).

[0066] It should also be noted that, in this embodiment, even though the scale area corresponding to the pointer rotation point is obtained through the above method, the scale area has been stretched into a rectangular image after the second transformation and correction. Therefore, it cannot be read according to the parameters of the pointer's initial rotation. The pointer also needs to be transformed relative to the stretching of the scale area, requiring the determination of the distance information between the key points at both ends of the scale area, which is the length of the rectangular image corresponding to the scale area (determined by a change ratio in this embodiment). Specifically, the above image processing is performed based on the preset pointer rotation point and the key points at both ends of the scale area to generate pointer rotation information. Angle mapping is then performed based on this pointer rotation information to determine the mapping data corresponding to the pointer rotation. (See [reference needed]). Figure 3 ,include:

[0067] S310: Determine the minimum and maximum rotation angles of the pointer relative to the horizontal direction based on the key points at both ends of the scale area;

[0068] In the above steps, the minimum rotation angle is the angle of rotation of the pointer rotation point from the starting point of the scale area to the key point in the horizontal direction, denoted as αmin, and the maximum rotation angle is the angle of rotation of the pointer rotation point from the ending point of the scale area to the key point in the horizontal direction, denoted as αmax.

[0069] S320: Determines the actual rotation angle of the pointer relative to the horizontal direction based on a preset pointer rotation point through image processing;

[0070] In the above steps, the actual rotation angle is the angle of rotation of the pointer rotation point relative to the horizontal direction, which is obtained through image processing, and is denoted as α.

[0071] S330: Calculate the first difference between the actual rotation angle and the minimum rotation angle, calculate the second difference between the maximum rotation angle and the minimum rotation angle, and determine the mapping data corresponding to the pointer rotation based on the ratio of the first difference and the second difference.

[0072] As can be seen, the above mapping data should actually be understood as the ratio of the pointer rotation change to the total rotatable change, i.e., the rotation ratio. Therefore, it can be expressed by the following expression: X = β / (αmax - αmin); X is the mapping data, β = α - αmin, which determines the pointer change data so as to determine the length of the pointer change in the scale area of ​​the converted rectangular image.

[0073] S400: Determine the position of the pointer under the corresponding scale interval of the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, so as to obtain the target reading corresponding to the pointer.

[0074] In this embodiment, it should be noted that, in addition to the key point detection and transformation correction mentioned above, it also includes the target detection results for each scale value. Generally, the scale lines of a pointer-type instrument panel are linear, meaning each scale interval is equal; otherwise, they are non-linear, meaning the change within two adjacent scale intervals is linear, but the change in the marked scale value is not linear (e.g., scale value changes such as 0, 40, 70, 90, 100). The pointer change ratio determined in this embodiment is applicable to both linear and non-linear change scenarios. Specifically, the position of the pointer under the rectangular image is determined based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area to obtain the target reading corresponding to the pointer. (See [reference needed]). Figure 4 ,include:

[0075] S410: Obtain the first endpoint and the second endpoint at both ends of the scale interval corresponding to the scale area where the pointer is located; obtain the first scale value and the second scale value corresponding to the first endpoint and the second endpoint, respectively;

[0076] In the above steps, the two endpoints corresponding to the scale interval where the pointer is located are determined through the image processing or semantic recognition model. For example, based on the target detection results of the scale interval, the scale values ​​corresponding to different scale intervals and the range N of the dashboard scale values ​​are determined. As mentioned above, based on the mapping data X and the proportional relationship of each scale value relative to the scale area after transformation and correction, it is determined that the pointer falls within a certain small scale interval.

[0077] It should be noted that the above-mentioned scale values ​​can be obtained by the target detection module after detecting each scale interval, and then performing text recognition. The recognized numbers are the scale values.

[0078] S420: Obtain the total length data of the rectangular image corresponding to the scale area; determine the first position data based on the length of the first endpoint from the key point at the beginning of the scale area and the total length data; determine the second position data based on the length of the second endpoint from the key point at the beginning of the scale area and the total length data; determine the relationship between the scale value and position change of the pointer in the scale interval based on the scale change of the first scale value and the second scale value relative to the position change of the first position data and the second position data;

[0079] Based on the above example, let the first scale value of the first endpoint and the second scale value of the second endpoint be N1 and N2 respectively, and the first position data and the second position data be X1 and X2 respectively. The above first position data (second position data) is the position of the first endpoint (second endpoint) on the rectangular image corresponding to the scale area. Then, the relationship between the scale and position change in the scale interval where the pointer is located is (N2-N1) / (X2-X1).

[0080] S430: Based on the relationship between the scale and the position change, and the change of the mapping data corresponding to the pointer rotation relative to the first position data / second position data, determine the scale change of the pointer relative to the first scale value / second scale value, so as to obtain the target reading of the pointer.

[0081] In the above steps, in order to determine the actual position of the pointer in the scale interval, it is necessary to determine the change of the pointer relative to the first position data / second position data. Therefore, based on the above example, it can be calculated according to the following formula: num=N1+(X-X1)*((N2-N1)) / ((X2-X1)), where X is the mapped data and num is the target reading.

[0082] Based on the above, this embodiment can be used for both uniform and non-uniform distribution of scale values ​​in various scale intervals. It should be noted that when the scale values ​​in each scale interval are uniformly distributed, determining the pointer's corresponding position is relatively convenient. Specifically, when the scale values ​​corresponding to each scale interval on the dashboard are uniformly distributed, determining the pointer's position under the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area to obtain the target reading of the pointer includes: obtaining the scale value at the beginning and end of the scale area to calculate the scale change range; determining the change of the pointer relative to the beginning scale value of the scale area based on the product of the scale change range and the mapping data, and obtaining the target reading of the pointer. That is, based on the target detection results of the scale values, the maximum value Nmax (i.e., the scale value corresponding to the key point at the end) and the minimum value Nmin (i.e., the scale value corresponding to the beginning) of the dashboard scale values ​​can be determined, and the final reading is: num = Nmin + X * (Nmax - Nmin), where X is the mapping data.

[0083] This implementation uses a keypoint detection model to achieve keypoint detection and localization in steps S200-S400. This model has high accuracy requirements for localizing S / 2-1 keypoints, but lower accuracy requirements for scale value target detection. Therefore, as an example, the following model structure can be adopted: feature extraction is performed on the image to be processed using a channel expansion layer and three feature extraction layers. Then, feature maps of different scales are extracted through an FPN layer. After keypoint localization is performed by multiple keypoint detection heads, the keypoint prediction results are output after being connected by a fully connected layer. At the same time, another target detection network is output at the FPN output to perform scale value target detection. Since the scale value images are very small and the scale value text recognition is easy, a simple MobileNetV1 network with only 8 layers is used, which is trained under supervision using CTC-loss.

[0084] The dashboard reading recognition method provided in this embodiment has better accuracy in reading pointer dashboard values ​​and is robust to pointer dashboard images with tilt angles. The key point detection method based on a multi-key point detection head can achieve data reading of pointer dashboards with arbitrary curvature and size with high accuracy. The models used in each step are decoupled from each other, so they can be flexibly replaced according to the target scenario. Even pointer dashboards with non-linear scales can achieve high accuracy reading. It is easy to operate and has a wide range of applications, including automatic and accurate reading of dashboards with curved scales and uneven scale distribution.

[0085] Example 2: The present invention also provides an instrument panel reading recognition device 5, see reference. Figure 5 ,include:

[0086] Preprocessing module 51 is used to perform key point detection and preprocessing on the acquired dashboard image to obtain an image to be processed containing the dashboard scale area and pointer.

[0087] In the above module, the transformation correction includes stretching, scaling, and cropping based on pre-set reference points. Specifically, as an example, a simple modified target detection model based on YOLOv5s can be used to detect dashboards in the natural environment, obtain their reference points, and then obtain a corrected standard image to achieve the aforementioned preprocessing.

[0088] The scale region correction module 52 is used to locate each scale interval in the scale region in the image to be processed by key points, and to perform transformation correction on the scale region based on each key point, so that the image corresponding to the scale region is converted into a rectangular image.

[0089] In the above module, S key points (the value of S is related to the number of scale intervals marked on the dashboard) that can contain complete scale lines are extracted. The function of these key points is to describe the positional information of each scale line. S key points are set and grouped according to the order of the scale values. Four key points are used to locate the four ends of each scale interval. After transformation and correction with a preset correction ratio, the images are stitched together according to the sequence marks, so that the image of the scale area is converted into a rectangular image. The key points at both ends of the scale lines shared by adjacent intervals will coincide. Therefore, it is possible for two adjacent scale intervals to share two key points on one end of the stitching.

[0090] The pointer rotation recognition module 53 is used to perform image processing based on the preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information, and to determine the corresponding mapping data of pointer rotation by performing angle mapping based on the pointer rotation information.

[0091] In the above module, a pointer rotation point can be preset, which represents the position of pointer rotation. The point around which the pointer rotates is recorded as point 0. Based on the preset pointer rotation point and two key points at both ends of the scale area (start and end), the rotation angle when the pointer rotates around point 0 and points to the scale is calculated, and the horizontal distance of the starting end of the scale area corresponding to the pointer being displayed as a rectangular image is determined (by changing the ratio).

[0092] The calculation module 54 is used to determine the position of the pointer under the corresponding scale interval of the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, so as to obtain the target reading corresponding to the pointer.

[0093] In the above calculation module, the two endpoints corresponding to the scale interval where the pointer is located are determined, the relationship between the scale and position change of the scale interval where the pointer is located is determined, and the change of the pointer relative to the corresponding endpoint of the scale interval is determined by the calculated mapping data corresponding to the pointer rotation in order to calculate the target reading.

[0094] The instrument panel reading recognition method provided in this embodiment has better accuracy in reading numerical values ​​of pointer instrument panels. It has good robustness to pointer instrument panel images with tilt angles. Based on the key point detection method of multi-key point detection head, it can realize data reading of pointer instrument panels with arbitrary curvature and size, with high accuracy and simple operation. It can be applied to the automatic and accurate reading of instrument panels with curved scales and uneven scale distribution without manual operation.

[0095] Example 3: To achieve the above objectives, the present invention also provides a computer device 6, see reference. Figure 6The computer device may include multiple computer devices, and the system components implementing the dashboard reading recognition method of Embodiment 1 may be distributed across different computer devices 6. Computer devices 6 may be smartphones, tablets, laptops, desktop computers, etc., executing programs. The computer device in this embodiment includes, but is not limited to: a memory 61, a processor 62, and a cache for the dashboard reading recognition method of Embodiment 1, all of which are interconnected via a system bus. It should be noted that... Figure 6 Only computer devices with components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0096] In this embodiment, the memory 61 may include a program storage area and a data storage area. The program storage area may store the application programs required by the operating system and at least one function; the data storage area may store user data information on the computer device. In some embodiments, the processor 62 may be a central processing unit (CPU), a controller, or other data processing chip. The processor 62 is typically used to control the overall operation of the computer device. In this embodiment, the processor 62 is used to run the program code or data stored in the memory 61, for example, to run the system to implement the dashboard reading recognition method of Embodiment 1.

[0097] It should be noted that only computer device 6 with components 61-62 is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0098] Example 4: To achieve the above objectives, the present invention also provides a computer-readable storage medium, which includes multiple storage media such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., storing computer programs thereon. When the program is executed by processor 62, it implements corresponding functions. In this embodiment, the computer-readable storage medium is used to store the TSC-DDPG model algorithm, and when executed by processor 62, it implements the dashboard reading recognition method of Example 1 and the dashboard reading recognition device of Example 2.

[0099] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for recognizing dashboard readings, characterized in that, include: After key point detection is performed on the acquired dashboard image, the first transformation correction is performed to obtain the image to be processed containing the dashboard scale area and pointer. The key point detection adopts the following model structure: the channel expansion layer and three feature extraction layers are used to extract features from the image to be processed. Then, the FPN layer is used to extract feature maps of different scales. After key point localization is performed by multiple key point detection heads, the key point prediction results are output after being connected by a fully connected layer. In the image to be processed, each scale interval in the scale region is located by key points, and a second transformation correction is performed on the scale region based on each key point, so that the image corresponding to the scale region is converted into a rectangular image. Image processing is performed based on the preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information. Angle mapping is performed based on the pointer rotation information to determine the corresponding mapping data for pointer rotation. The preset pointer rotation point is achieved as follows: the image to be processed is input into a semantic segmentation model or object detection model to obtain the semantic segmentation result of the pointer. In order to obtain the position of pointer rotation, a pointer rotation point is preset. Based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, the position of the pointer under the scale interval of the rectangular image is determined, so as to obtain the target reading corresponding to the pointer; The process utilizes a channel expansion layer and three feature extraction layers to extract features from the image to be processed. Then, it extracts feature maps of different scales through an FPN layer. After key point localization is performed by multiple key point detection heads, the key point prediction results are output after being connected by a fully connected layer. At the same time, another target detection network is output in the FPN to perform scale value target detection. In the image to be processed, each scale interval in the scale region is located using key points, and a second transformation correction is performed on the scale region based on each key point to convert the image corresponding to the scale region into a rectangular image, including: Each scale interval in the scale region is sequentially marked, wherein each scale interval includes two endpoints, and each endpoint corresponds to two ends; Four key points are used to locate the four ends of each scale interval to obtain quadrilateral sub-blocks corresponding to each scale interval, which are composed of each key point. After performing the second transformation correction on each quadrilateral sub-block, they are stitched together according to the sequence markings, so that the image of the scale area is converted into a rectangular image. The image processing based on the preset pointer rotation point and key points at both ends of the scale area generates pointer rotation information. Angle mapping is then performed based on this pointer rotation information to determine the corresponding mapping data for the pointer rotation, including: The minimum and maximum rotation angles of the pointer relative to the horizontal direction are determined based on the key points at both ends of the scale area. The actual rotation angle of the pointer relative to the horizontal direction is determined by image processing based on the preset pointer rotation point; Calculate the first difference between the actual rotation angle and the minimum rotation angle, calculate the second difference between the maximum rotation angle and the minimum rotation angle, and determine the mapping data corresponding to the pointer rotation based on the ratio of the first difference and the second difference; The step of determining the position of the pointer under the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, in order to obtain the target reading corresponding to the pointer, includes: Obtain the first and second endpoints of the corresponding scale interval under the scale area where the pointer is located; Obtain the first scale value and the second scale value corresponding to the first endpoint and the second endpoint, respectively; Obtain the total length data of the rectangular image corresponding to the scale area; The first position data is determined based on the length of the first endpoint from the key point at the beginning of the scale area and the total length data; the second position data is determined based on the length of the second endpoint from the key point at the beginning of the scale area and the total length data. Based on the scale changes of the first and second scale values ​​relative to the position changes of the first and second position data, the relationship between scale and position changes within the scale interval where the pointer is located is determined. Based on the relationship between the scale and the position change, and the change of the mapping data corresponding to the pointer rotation relative to the first position data / second position data, the scale change of the pointer relative to the first scale value / second scale value is determined, so as to obtain the target reading of the pointer.

2. The identification method according to claim 1, characterized in that: Two key points are shared at the end of the splicing side of two adjacent scale intervals.

3. The identification method according to claim 1, characterized in that, When the scale values ​​corresponding to each of the scale intervals are evenly distributed, the step of determining the position of the pointer below the scale interval corresponding to the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, in order to obtain the target reading corresponding to the pointer, includes: Obtain the scale values ​​at the beginning and end of the scale area to calculate the scale variation range; The change in the pointer relative to the starting point of the scale area is determined by multiplying the scale change range with the mapping data, and the target reading corresponding to the pointer is obtained.

4. The identification method according to claim 1, characterized in that: The transformation correction involves performing one or more operations, such as stretching, scaling, and cropping, on the acquired dashboard image after key point detection.

5. A dashboard reading recognition device, characterized in that, include: The preprocessing module is used to perform key point detection and first transformation correction on the acquired dashboard image to obtain the image to be processed containing the dashboard scale area and pointer. The key point detection adopts the following model structure: the channel expansion layer and three feature extraction layers are used to extract features from the image to be processed. Then, the FPN layer extracts feature maps of different scales. After key point localization is performed by multiple key point detection heads, the key point prediction results are output after being connected by a fully connected layer. The scale region correction module is used to locate each scale interval in the scale region in the image to be processed by key points, and to perform a second transformation correction on the scale region based on each key point, so that the image corresponding to the scale region is converted into a rectangular image. The pointer rotation recognition module is used to perform image processing based on a preset pointer rotation point and key points at both ends of the scale area to generate pointer rotation information. The pointer rotation information is used to perform angle mapping to determine the corresponding mapping data of pointer rotation. The preset pointer rotation point is achieved as follows: the image to be processed is input into a semantic segmentation model or object detection model to obtain the semantic segmentation result of the pointer. In order to obtain the position of pointer rotation, a pointer rotation point is preset. The calculation module is used to determine the position of the pointer under the corresponding scale interval of the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, so as to obtain the target reading corresponding to the pointer; The process utilizes a channel expansion layer and three feature extraction layers to extract features from the image to be processed. Then, it extracts feature maps of different scales through an FPN layer. After key point localization is performed by multiple key point detection heads, the key point prediction results are output after being connected by a fully connected layer. At the same time, another target detection network is output in the FPN to perform scale value target detection. In the image to be processed, each scale interval in the scale region is located using key points, and a second transformation correction is performed on the scale region based on each key point to convert the image corresponding to the scale region into a rectangular image, including: Each scale interval in the scale region is sequentially marked, wherein each scale interval includes two endpoints, and each endpoint corresponds to two ends; Four key points are used to locate the four ends of each scale interval to obtain quadrilateral sub-blocks corresponding to each scale interval, which are composed of each key point. After performing the second transformation correction on each quadrilateral sub-block, they are stitched together according to the sequence markings, so that the image of the scale area is converted into a rectangular image. The image processing based on the preset pointer rotation point and key points at both ends of the scale area generates pointer rotation information. Angle mapping is then performed based on this pointer rotation information to determine the corresponding mapping data for the pointer rotation, including: The minimum and maximum rotation angles of the pointer relative to the horizontal direction are determined based on the key points at both ends of the scale area. The actual rotation angle of the pointer relative to the horizontal direction is determined by image processing based on the preset pointer rotation point; Calculate the first difference between the actual rotation angle and the minimum rotation angle, calculate the second difference between the maximum rotation angle and the minimum rotation angle, and determine the mapping data corresponding to the pointer rotation based on the ratio of the first difference and the second difference; The step of determining the position of the pointer under the rectangular image based on the mapping data corresponding to the pointer rotation and the rectangular image corresponding to the scale area, in order to obtain the target reading corresponding to the pointer, includes: Obtain the first and second endpoints of the corresponding scale interval under the scale area where the pointer is located; Obtain the first scale value and the second scale value corresponding to the first endpoint and the second endpoint, respectively; Obtain the total length data of the rectangular image corresponding to the scale area; The first position data is determined based on the length of the first endpoint from the key point at the beginning of the scale area and the total length data; the second position data is determined based on the length of the second endpoint from the key point at the beginning of the scale area and the total length data. Based on the scale changes of the first and second scale values ​​relative to the position changes of the first and second position data, the relationship between scale and position changes within the scale interval where the pointer is located is determined. Based on the relationship between the scale and the position change, and the change of the mapping data corresponding to the pointer rotation relative to the first position data / second position data, the scale change of the pointer relative to the first scale value / second scale value is determined, so as to obtain the target reading of the pointer.

6. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the dashboard reading recognition method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the instrument panel reading recognition method according to any one of claims 1-4.

Images