Data identification method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a data recognition method and device, equipment and a storage medium, and the method comprises the steps: carrying out the scale recognition of an obtained target image, and obtaining the attribute information of a plurality of scales; pixel segmentation is carried out on the target image, and the pixel position of a pointer and the pixel positions of the scales are determined respectively; based on the attribute information of the multiple scales and the pixel positions of the multiple scales, determining matching information between the attribute information of the multiple scales and the pixel positions; based on the pixel position of the pointer and the pixel positions of the scales, determining a target scale matched with the pointer in the scales; and determining data information in the target image based on the matching information of the target scale and the pixel position of the pointer.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image recognition technology, and relate to but are not limited to a data recognition method, device, equipment and storage medium. Background Art

[0002] Industrial equipment uses a variety of meters to display the status of the equipment. With the development of technology, remote transcription is achieved through video automation monitoring technology, but it is not possible to read meters in real time 24 hours a day, and image-based data (for example, the recognition of meter counts) recognition methods are needed. In related technologies, data recognition methods based on image analysis have large errors and require template matching. Data recognition methods based on artificial intelligence technology use target detection and semantic segmentation to achieve readings, but for rotated or skewed images, data recognition has certain errors. Summary of the invention

[0003] The embodiment of the present application provides a data identification technology solution.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a data identification method, the method comprising:

[0006] Performing scale recognition on the acquired target image to obtain attribute information of multiple scales;

[0007] Performing pixel segmentation on the target image to respectively determine the pixel position of the pointer and the pixel positions of the multiple scales;

[0008] Based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales, determining matching information between the attribute information of the plurality of scales and the pixel positions;

[0009] Based on the pixel position of the pointer and the pixel positions of the plurality of scales, determining a target scale matching the pointer among the plurality of scales;

[0010] Based on the matching information of the target scale and the pixel position of the pointer, data information in the target image is determined.

[0011] In a second aspect, an embodiment of the present application provides a data identification device, the device comprising:

[0012] A first recognition module is used to perform scale recognition on the acquired target image to obtain attribute information of multiple scales;

[0013] A first segmentation module, used for performing pixel segmentation on the target image to respectively determine the pixel position of the pointer and the pixel positions of the plurality of scales;

[0014] A first determination module, configured to determine matching information between the attribute information of the plurality of scales and the pixel positions of the plurality of scales based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales;

[0015] A second determination module, configured to determine a target scale matching the pointer from among the multiple scales based on the pixel position of the pointer and the pixel positions of the multiple scales;

[0016] The third determination module is used to determine the data information in the target image based on the matching information of the target scale and the pixel position of the pointer.

[0017] An embodiment of the present application provides a computer storage medium, on which computer executable instructions are stored. After the computer executable instructions are executed, the above-mentioned data identification method can be implemented.

[0018] An embodiment of the present application provides a terminal device, which includes a memory and a processor, wherein the memory stores computer executable instructions, and the processor can implement the above-mentioned data identification method when running the computer executable instructions on the memory.

[0019] An embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, implements some or all of the steps in the above method.

[0020] In an embodiment of the present application, the acquired target image is scale-recognized to obtain the attribute information of multiple scales in the target image; and the target image is pixel-segmented to segment the pixel position of the pointer and the pixel position of multiple scales; thus, by performing scale-recognition and pixel-segmentation on the target image, the attributes, pixel positions and pixel positions of multiple scales can be accurately obtained, which is convenient for the subsequent establishment of matching between the pointer and multiple scales. Afterwards, according to the attribute information of multiple scales and the pixel positions of the multiple scales, the matching information between the attribute information and the pixel positions of the multiple scales is determined; thus, for each scale, the attribute information and the position information of the scale are matched to obtain the matching information image of the scale, that is, each matching information can characterize the binding of the attribute information and the pixel position of a scale. According to the pixel position of the pointer and the pixel positions of multiple scales, the target scale matching the pointer is determined in the multiple scales, and according to the matching information of the target scale and the pixel position of the pointer, the data information in the target image is determined. In this way, through the pixel position of the pointer and the pixel positions of multiple scales, the target scale whose pixel position matches the pointer can be found among the multiple scales. Then, by combining the matching information of the target scale and the pixel position of the pointer, the value indicated by the pointer can be quickly determined, thereby improving the accuracy of the recognized data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0022] Figure 1 A schematic diagram of the implementation flow of the data identification method provided in the embodiment of the present application;

[0023] Figure 2A A schematic diagram of another implementation flow of the data identification method provided in an embodiment of the present application;

[0024] Figure 2B A schematic diagram of another implementation flow of the data identification method provided in the embodiment of the present application;

[0025] Figure 3 A schematic diagram of an application scenario of the data identification method provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of another application scenario of the data identification method provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of another application scenario of the data identification method provided in the embodiment of the present application;

[0028] Figure 6A schematic diagram of the structure of a data identification device provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0034] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are first described.

[0035] 1) Deep learning is to learn the inherent laws and representation levels of sample data. The information obtained in the learning process is very helpful for interpreting data such as text, images and sounds. The ultimate goal is to enable machines to have analytical learning capabilities like humans and be able to recognize data such as text, images and sounds.

[0036] 2) Computer vision refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further process images so that the computer processes them into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data.

[0037] The present application embodiment provides a data identification method, which is applied to a terminal device. The terminal device includes but is not limited to a mobile phone, a laptop computer, a tablet computer, a mobile Internet device, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other devices with data processing capabilities. The function implemented by the method can be implemented by calling a program code through a processor in the terminal device, and of course the program code can be stored in a computer storage medium. It can be seen that the terminal device includes at least a processor and a storage medium. The processor can be used to execute the data identification method, and the memory can be used to store the data required and the data generated during the execution of the data identification method. Figure 1 A schematic diagram of the implementation flow of the data identification method provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0038] Step S101, performing scale recognition on the acquired target image to obtain attribute information of multiple scales.

[0039] In some embodiments, the target image may be a meter image, which may be an image of a screen including a meter in any scene, or may be obtained by performing meter detection on an original image and cropping the meter detection frame. The meter image is an image with high picture quality, for example, a meter image obtained by performing image correction on the meter in the screen. The meter is a device for detecting the status of the device, for example, a device installed in the industrial device itself to display the status of the device.

[0040] In some possible implementations, a meter image is used for illustration. After acquiring the meter image, the scales in the meter image are identified to obtain the attribute information of each scale in the meter. The attribute information is used to describe the content presented by the scale in the meter, including the position of the scale value in the meter, the scale value corresponding to the scale, etc.

[0041] Step S102, performing pixel segmentation on the target image to respectively determine the pixel position of the pointer and the pixel positions of the multiple scales.

[0042] In some embodiments, the target image may be a meter image, and the pixel position of the pointer and the pixel position of the multiple scales represent the position of the pointer in the image at the pixel level, and represent the position of the multiple scales in the image at the pixel level, respectively. The acquired meter image is pixel-segmented, and the pixel positions of different objects in the meter image are distinguished to obtain a detection frame of the pixel position of the pointer and a detection frame of the pixel position of the multiple scales, and the pixel position of the pointer and the pixel position of the multiple scales are obtained through these detection frames. By segmenting the pixel positions of the pointer and the multiple scales at the pixel level, the positions of the pointer and the multiple scales in the meter image can be more accurately reflected.

[0043] In some possible implementations, a meter image is used for illustration, and a meter segmentation model based on deep learning is used to perform pixel segmentation on the meter image to determine the pixel position of the pointer and the pixel position of the multiple scales, respectively; in this way, the positions of the pointer and the multiple scales in the meter image can be segmented more quickly and accurately, so as to determine the distribution of the scales in the meter image in combination with the scale numerical positions.

[0044] Step S103: determining matching information between the attribute information of the plurality of scales and the pixel positions based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales.

[0045] In some embodiments, the attribute information and pixel positions of multiple scales are matched, so that the matching information of the attribute information and the pixel position of each scale can be obtained. In this way, each scale has the matching information of the attribute information and the pixel position.

[0046] In some possible implementations, for each scale, since the attribute information of the scale includes the scale numerical position, any scale numerical position is compared with any pixel position, and if the distance between the scale numerical position and the pixel position matches, for example, the scale numerical position and the pixel position overlap or nearly overlap, then it is determined that the scale numerical position and the pixel position belong to the same scale, so the pixel position and the scale numerical position are used as a piece of matching information of the corresponding scale. In this way, a piece of matching information includes the attribute information and the pixel position of the corresponding scale.

[0047] Step S104: determining a target scale matching the pointer from among the multiple scales based on the pixel position of the pointer and the pixel positions of the multiple scales.

[0048] In some embodiments, the pixel position of the pointer is compared with the pixel position of each scale, and a scale whose pixel position is closer to the pixel position of the pointer is determined from multiple scales. This scale is the target scale that matches the pointer.

[0049] In some possible implementations, by determining the distance between the pixel position of the pointer and the pixel position of each scale, the scale line closer to the position of the pointer is used as the target scale line. For example, the target scale line may be the two scale lines closest to the left and right sides of the pointer.

[0050] Step S105 , determining data information in the target image based on the matching information of the target scale and the pixel position of the pointer.

[0051] In some embodiments, the target image may be a meter image, and the data information may be a meter reading. By combining the matching information of the target scale and the pixel position of the pointer, the positional relationship between the pointer and the target scale can be determined, and the positional relationship can characterize the ratio of the position indicated by the pointer between the target scales. The value indicated by the pointer in the meter can be determined by combining the attribute information of the target scale in the matching information of the target scale, thereby obtaining the meter reading in the meter image.

[0052] In some possible implementations, a meter image is used for illustration. Since the matching information of the target scale includes the attribute information and pixel position of the target scale, the angle between the target scale and the extended line of the pointer is determined by combining the matching information of the target scale with the pixel position of the pointer. The value indicated by the pointer between the target scales can be analyzed by combining the attribute information and pixel position of the target scale, that is, the meter indication in the meter image is obtained.

[0053] In an embodiment of the present application, by performing scale recognition and pixel segmentation on the target image, the attributes, pixel positions and pixel positions of multiple scales can be accurately obtained, which is convenient for the subsequent establishment of a match between the pointer and multiple scales. For each scale, the attribute information and position information of the scale are matched to obtain the matching information image of the scale, that is, each matching information can characterize the binding of the attribute information and pixel position of a scale. Through the pixel position of the pointer and the pixel positions of multiple scales, the target scale whose pixel position matches the pointer can be found in multiple scales, so that by combining the matching information of the target scale with the pixel position of the pointer, the value indicated by the pointer can be quickly determined, thereby improving the accuracy of identifying the data information in the target image.

[0054] In some embodiments, the target image may be a meter image, and the attribute information of the scale may be quickly obtained by detecting the scale value of the meter image and identifying the scale value. That is, the above step S101 may be performed by Figure 2A The steps shown achieve:

[0055] Step S201 , performing scale value detection in the target image to obtain scale value positions of the multiple scales.

[0056] In some embodiments, the target image may be a meter image, and the scale value positions of the multiple scales, that is, the positions of the scale values ​​of the multiple scales in the meter image, can also represent the positions of the scales in the meter image. The trained scale value detection model is used to detect the scale values ​​in the meter image, and a rectangular detection frame identifying the scale values ​​in the image is output, so as to obtain the positions of the rectangular detection frames corresponding to the multiple scales, that is, the positions of the multiple scale values ​​in the marked image. The trained scale value detection model can be trained through the following process: first, a sample meter image is obtained; second, the positions of the scale values ​​in the sample meter image are marked with rectangular frames, and the positions of all rectangular frames in the sample meter image are output; finally, the sample meter image with annotations is trained using a deep learning framework to obtain a scale value detection model.

[0057] Step S202, performing scale value recognition on the scale at each scale value position to obtain the scale values ​​of the multiple scales.

[0058] In some embodiments, the target image may be a meter image, and the trained scale value recognition model is used to recognize the scale values ​​in the meter image, and the scale values ​​in the image are output, so as to obtain the scale values ​​of multiple scales. The trained scale value recognition model may be trained through the following process: first, using the detection frame of the scale value position to obtain a scale value position image; second, according to the scale value position image, the corresponding digital category annotation image is output; finally, the scale value position image with digital category annotation is trained using a deep learning framework to obtain a trained scale value recognition model.

[0059] Step S203: determining the scale value positions and the scale values ​​of the multiple scales as the attribute information.

[0060] In some embodiments, for each scale, the scale value position and the scale value at the position are in a corresponding relationship, so that the scale value position of a scale and the scale value at the position are used as attribute information of the scale.

[0061] In an embodiment of the present application, the scale value is detected on the target image to obtain the scale value position of each scale, and the scale value is identified to obtain the scale value of the scale; in this way, the scale value position of each scale in the target image can be detected and the scale value of each scale can be identified, thereby quickly and accurately obtaining attribute information including the scale value position and the scale value.

[0062] In some embodiments, by performing pixel segmentation on the target image at the pixel level, the pixel positions of the pointer and the scale can be accurately obtained, that is, the above step S102 can be implemented by the following steps S121 and S122 (not shown):

[0063] Step S121, determining the pixel level of the target image.

[0064] In some embodiments, the pixel level of the target image may be the color components forming the target image. For example, if the color components forming the target image are red, green, and blue (RGB), then the pixel level of the target image is RGB.

[0065] In some possible implementations, the target image is a meter image for illustration, and the color component of the meter image, that is, the pixel level of the meter image, can be obtained by performing image analysis on the meter image. Alternatively, the pixel level carried by the meter image itself can be obtained.

[0066] Step S122, segmenting the pointer and the multiple scales in the target image using the pixel level as a segmentation unit to obtain the pixel position of the pointer and the pixel positions of the multiple scales.

[0067] In some embodiments, the target image may be a meter image, and the meter image is pixel-segmented at the pixel level, so that the positions of different objects in the meter image can be represented by the formation of pixels. Since different scales correspond to different pixels, the pixel positions obtained after segmentation at the pixel level are also different. Therefore, after segmenting the pointer and multiple scales in the meter image at the pixel level, the pixel positions of the pointer and the pixel positions of the multiple scales obtained are different.

[0068] In some possible implementations, a meter image is used for illustration, and the pointer and the multiple scales in the meter image are segmented using a trained meter segmentation model to obtain the pixel position of the pointer and the pixel position of the multiple scales. The trained meter segmentation model can be trained through the following process: first, the pointer and scale lines in the meter image are annotated at the pixel level, classified, and the annotated image is output; then the annotated meter image is trained using a deep learning framework to obtain a trained meter segmentation model.

[0069] Through the above steps S121 and S122, by analyzing the pixel level of the target image, the positions of the pointer and scales in the target image are segmented according to the pixel level to accurately output the positions of the pointer and scales represented by pixels, that is, the pixel positions of the pointer and the pixel positions of the multiple scales are obtained, so that the pixel positions of the segmented pointer and scales are more accurate.

[0070] In some embodiments, by associating the scale value, scale value position and pixel position of a scale, the matching information of the scale is obtained, that is, the above step S103 can be implemented by the following steps S131 and S132 (not shown in the figure):

[0071] Step S131, determining a first distance between a scale value position in the attribute information of the plurality of scales and the pixel position.

[0072] In some embodiments, since the attribute information of the scale includes the scale numerical position and the scale numerical value, wherein the scale numerical position can represent the position of the scale, the first distance is obtained by determining the distance between each scale numerical position and each pixel position, so that it is convenient to judge whether the scale numerical position and the pixel position involved in calculating the first distance belong to the same scale through the first distance.

[0073] Step S132, when the first distance is less than a preset distance threshold, the scale value position corresponding to the first distance, the pixel position and the scale value corresponding to the scale value position are associated to obtain matching information of the scale corresponding to the first distance.

[0074] In some embodiments, when the first distance is less than a preset distance threshold, it means that the scale numerical position and pixel position corresponding to the first distance belong to the same scale, so the scale numerical position and pixel position corresponding to the first distance are used as intermediate matching information. Afterwards, in the intermediate matching information of each scale, the scale numerical value and pixel position corresponding to the numerical position are associated again to obtain the matching information of the same scale. By analogy, for each scale, matching information in which the scale numerical position, pixel position and scale numerical value are associated together can be obtained. In this way, by determining the distance between the scale numerical position and the pixel position, if the distance is relatively small, it means that the pixel position of the scale divided by pixel level belongs to the same scale as the detected scale numerical position, so that the scale numerical position, pixel position and scale numerical value of the same scale are associated together to form matching information, so that the scale numerical value of the scale can be quickly obtained through the matching information later, and then the indication of the pointer can be quickly determined.

[0075] In some embodiments, according to the pixel positions of the pointer and the multiple scales, a target scale that is closer to the pointer is determined from the multiple scales, that is, the above step S104 can be implemented by the following steps S141 and S142 (not shown):

[0076] Step S141, determining a second distance between the pixel position of the pointer and the pixel position of each of the multiple scales.

[0077] In some embodiments, based on the pixel position of the pointer and the pixel positions of the multiple scales, the second distance between the pointer and each scale is determined. Thus, the second distance can indicate which scale is closer to the pointer, and thus determine which scales are adjacent to the pointer.

[0078] Step S142: Based on the second distance, determine the scale adjacent to the pointer as the target scale.

[0079] In some embodiments, by comparing the second distances between multiple scales and the pointer, the two scales with the closest second distance and adjacent to the pointer are determined as target scales; thus, the two scales adjacent to the pointer can be accurately and quickly determined. Thus, by determining the distance between the pixel position of the pointer and the pixel position of each scale, the target scales on the left and right sides of the pointer and closest to the pointer can be quickly selected, so that the value indicated by the pointer can be accurately and conveniently analyzed through the scale values ​​of the target scales adjacent to the pointer.

[0080] In some embodiments, the pointer reading is quickly determined by the angle between the target scale and the pointer and the value of the target scale, that is, the above step S105 can be performed by Figure 2B The steps shown achieve:

[0081] Step S221: determining the angle information between the target scale and the pointer based on the pixel position of the target scale and the pixel position of the pointer in the matching information.

[0082] In some embodiments, after determining the target scale of the pointer, the line where the target scale and the pointer are located is extended according to the pixel position of the target scale and the pixel position of the pointer, so that the extended line of the target scale and the extended line of the pointer intersect at one point, and then the angle between the target scale and the pointer can be determined through the intersection point to obtain angle information.

[0083] Step S222: determining the indicated value of the pointer based on the angle information and the scale value of the target scale in the matching information.

[0084] In some embodiments, after obtaining the angle information, the ratio between the scales on the left and right sides of the pointer can be known; thus, according to the ratio and the scale value of the target scale, the indicated value of the pointer can be quickly determined.

[0085] In some possible implementations, the target scale includes two scales adjacent to the pointer, and the value indicated by the pointer can be obtained by combining the ratio between the angles and the scale values, that is, the above step S222 can be implemented by the following steps S231 and S232 (not shown in the figure):

[0086] Step S231, determining the ratio between the angle information of the two scales.

[0087] Here, the target scale is the two nearest scales on the left and right sides of the pointer. After determining the angle information between the two scales and the pointer, the ratio between the two angle information is further determined, which can represent the ratio of the value indicated by the pointer to the scale values ​​of the two scales.

[0088] Step S232, determining the indicated value of the pointer based on the ratio and the scale value of each of the two scales.

[0089] Here, since the ratio can represent the proportion of the indicated value of the pointer in the scale values ​​of the two scales, the indicated value of the pointer can be quickly obtained by combining the ratio with the scale values. In this way, by analyzing the ratio between the angles formed by the pointer and the two scales, the proportion of the value of the pointer in the scale values ​​of the two scales can be clearly determined, so the indicated value of the pointer can be quickly located by the scale value of the target scale.

[0090] Step S223: determining the indicated value of the pointer as the data information in the target image.

[0091] Here, the target image can be a meter image, and the obtained pointer indication value is the meter reading in the meter image. Thus, by analyzing the angle information between the target scale and the pointer, the proportion of the pointer between the target scales can be known, and thus, combined with the scale value of the target scale, the meter reading in the meter image can be accurately determined with low computational complexity.

[0092] In some embodiments, a meter image is used for illustration. By correcting the image of the meter marked by the detection frame, a meter image with higher image quality can be obtained. This can be achieved through the following process:

[0093] The first step is to obtain the original image of the screen including the meter.

[0094] In some embodiments, the original image is an image in any scene including a meter and requiring meter indication recognition, and may be an image with simple content or an image with complex content. In some possible implementations, the original image is obtained by capturing an image of the meter; or the original image may be an image including a meter sent by other devices.

[0095] The second step is to perform meter detection on the original image to obtain a detection frame of the meter.

[0096] In some embodiments, the meter image is detected in the original image, thereby outputting a detection frame identifying the meter, so that the position of the detection frame is the position of the meter in the original image. In some possible implementations, the meter detection is performed on the original image using a trained meter detection model to obtain a detection frame of the meter.

[0097] The third step is to perform image correction on the image in the detection frame to obtain the meter image.

[0098] In some embodiments, the image in the detection frame is the image of the meter, so the image in the detection frame is corrected to correct the skewness and clarity of the text in the image. For example, the image in the detection frame is rotated so that the text is regular and non-skewed relative to the image. The image in the detection frame after such correction is used as the meter image. In some possible implementations, the meter correction model established by the deep learning framework is used to perform image correction on the image in the detection frame to obtain the meter image. In this way, by performing meter detection on the acquired original image and performing image correction on the image in the detection frame to obtain a meter image with higher picture quality, the meter reading can be recognized by the meter image, which can improve the accuracy of meter reading recognition.

[0099] Below, an exemplary application of an embodiment of the present application in an actual application scenario will be described, taking the identification of meter readings of industrial equipment as an example.

[0100] In some embodiments, meter indication recognition can be achieved by the following steps:

[0101] In the first step, a meter detection model is established using a deep learning framework to identify meters from images and output the coordinates of the detection box.

[0102] In some possible implementations, the process of training a deep learning-based meter detection model is as follows:

[0103] First, obtain original images from different perspectives, different time periods, and different scenes;

[0104] Here, the original image can be Figure 3 The image shown in Figure 3 The original image includes a dial 31, a pointer 32, multiple scales and scale values, etc., wherein the scale 34 represents two scales among the multiple scales.

[0105] Secondly, use rectangular boxes to mark the positions of meters in the image, and output a file with the position information of all rectangular boxes in the image and their corresponding category information;

[0106] like Figure 3 As shown, a rectangular frame 33 is used to mark the position of the meter in the image.

[0107] Finally, the deep learning framework is used to train the original images with labeled information files to obtain the meter detection model.

[0108] In the second step, the meter image is extracted from the original image according to the detection frame coordinates in the first step, and a text detection and recognition model is established using a deep learning framework to output the numerical value and coordinate frame information of the scale in the meter.

[0109] In some possible implementations, the text detection and recognition model includes: a scale value detection model and a scale value recognition model, wherein the scale value detection model based on deep learning is trained as follows:

[0110] First, using the rectangular frame of the meter obtained in the first step, the image of the meter is obtained from the original image;

[0111] Secondly, use rectangular boxes to mark the positions of scale values, and output files containing the position information of all rectangular boxes in the image and their corresponding category information;

[0112] Finally, the deep learning framework is used to train meter images with labeled information files to obtain the scale value detection model.

[0113] In some possible implementations, a scale value recognition model based on deep learning is trained. The specific process is as follows:

[0114] First, use the rectangular frame of the scale value in the second step to obtain an image of the scale value;

[0115] Secondly, according to the image of the scale value, the corresponding digital category annotation file is output;

[0116] Finally, a deep learning framework is used to train scale value images with digital category annotation files to obtain a scale value recognition model.

[0117] In the third step, based on the meter image taken in the second step, a meter segmentation model is established using a deep learning framework to output the pixel position and category of the pointer and dial scale.

[0118] Here, the pixel positions of the output pointer and dial scale are as follows: Figure 4 As shown in FIG. 8( a ), 41 represents the pixel position of the pointer, and 42 to 48 represent the pixel positions of the scale.

[0119] In some possible implementations, training a deep learning-based meter segmentation model is as follows:

[0120] First, using the rectangular frame of the meter in step 1, an image of the meter is obtained from the original image;

[0121] Then, use the annotation tool to annotate the pointer and each scale line in the meter image at the pixel level, classify them, and output the annotation information file;

[0122] Finally, the meter images with labeled information files are trained using a deep learning framework to obtain a meter segmentation model.

[0123] The fourth step is to match the dial scale and scale reading one by one, use the pointer and its two nearest scale lines to calculate the angle information of their intersection and connecting lines, and thus obtain the meter reading.

[0124] Here, the pointer and its two nearest scale lines are as follows Figure 4 As shown in Figure (b), the pointer is 41, and the two scale lines closest to the pointer 41 are 42 and 48. Figure 5 In the figure, the pointer 41, the scale line 42 and the scale line 48 intersect at a point 51, and the angle information of the angle between the pointer 41 and the scale line 42 and the angle information of the angle between the pointer 41 and the scale line 48 are determined.

[0125] In some possible implementations, the meter indication recognition is realized by using the pixels of the pointer, the pixels of the scale, the position of the scale frame and the value, including the following processes:

[0126] First, match the scale frame position with the scale pixel position, so that the scale pixel category corresponds to the scale value;

[0127] Secondly, fit the pointer and its two nearest scale lines to form three straight lines that extend and intersect at one point, and calculate the angle at which the pointer intersects the two scale lines;

[0128] Finally, the position pointed by the pointer is calculated using the two obtained angles and the values ​​of the two scale lines, thereby obtaining the data;

[0129] In an embodiment of the present application, text recognition is matched with semantic segmentation results to obtain the dial scale value, and the intersection of the pointer and the extension line of the two most adjacent scale lines is used to calculate the angle between the pointer and the adjacent scale lines, thereby calculating the data; thereby, the dial scale in the meter can be automatically obtained, so that the accuracy of data reading will be higher, thereby realizing the automatic reading of mechanical data on related equipment in substations, factories, and flammable and explosive places.

[0130] The present application embodiment provides a data identification device, Figure 6 A schematic diagram of the structure of the data identification device provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the data identification device 600 includes:

[0131] The first recognition module 601 is used to perform scale recognition on the acquired target image to obtain attribute information of multiple scales;

[0132] A first segmentation module 602, configured to perform pixel segmentation on the target image, and determine the pixel position of the pointer and the pixel positions of the plurality of scales respectively;

[0133] A first determination module 603, configured to determine matching information between the attribute information of the plurality of scales and the pixel positions of the plurality of scales based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales;

[0134] A second determination module 604 is configured to determine a target scale matching the pointer from among the multiple scales based on the pixel position of the pointer and the pixel positions of the multiple scales;

[0135] The third determination module 605 is used to determine the data information in the target image based on the matching information of the target scale and the pixel position of the pointer.

[0136] In some embodiments, the first identification module 601 includes:

[0137] A first detection submodule, configured to perform scale value detection in the target image to obtain scale value positions of the multiple scales;

[0138] A first identification submodule is used to identify the scale value of each scale value position to obtain the scale values ​​of the multiple scales;

[0139] The first determining submodule is used to determine the scale value positions and the scale values ​​of the multiple scales as the attribute information.

[0140] In some embodiments, the first segmentation module 602 includes:

[0141] A second determination submodule, used to determine the pixel level of the target image;

[0142] The first segmentation submodule is used to segment the pointer and the multiple scales in the target image using the pixel level as a segmentation unit to obtain the pixel position of the pointer and the pixel positions of the multiple scales.

[0143] In some embodiments, the first determining module 603 includes:

[0144] A third determination submodule, configured to determine a first distance between a scale value position in the attribute information of the plurality of scales and the pixel position;

[0145] The first association submodule is used to associate the scale value position corresponding to the first distance, the pixel position and the scale value corresponding to the scale value position to obtain matching information of the scale corresponding to the first distance when the first distance is less than a preset distance threshold.

[0146] In some embodiments, the second determining module 604 includes:

[0147] a fourth determination submodule, configured to determine a second distance between a pixel position of the pointer and a pixel position of each of the plurality of scales;

[0148] The fifth determining submodule is used to determine, based on the second distance, a scale adjacent to the pointer as the target scale.

[0149] In some embodiments, the third determination module 605 includes:

[0150] a sixth determination submodule, configured to determine angle information between the target scale and the pointer based on the pixel position of the target scale and the pixel position of the pointer in the matching information;

[0151] a seventh determination submodule, configured to determine the indicated value of the pointer based on the angle information and the scale value of the target scale in the matching information;

[0152] The eighth determination submodule is used to determine the indicated value of the pointer as the data information in the target image.

[0153] In some embodiments, the target scale includes two scales adjacent to the pointer, and the seventh determination submodule includes:

[0154] A first determining unit, used to determine the ratio between the angle information of the two scales;

[0155] The second determining unit is used to determine the indicated value of the pointer based on the ratio and the scale value of each of the two scales.

[0156] In some embodiments, the apparatus further comprises:

[0157] A first acquisition module is used to acquire an original image of a screen including a meter;

[0158] A first detection module, configured to perform meter detection on the original image to obtain a detection frame of the meter;

[0159] The first correction module is used to perform image correction on the image in the detection frame to obtain the target image.

[0160] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0161] It should be noted that in the embodiment of the present application, if the above-mentioned data identification method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a terminal device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a sports hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0162] The embodiment of the present application further provides a computer program product, which includes computer executable instructions. After the computer executable instructions are executed, the data identification method provided in the embodiment of the present application can be implemented.

[0163] An embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored. When the computer executable instructions are executed by a processor, the data identification method provided in the above embodiment is implemented.

[0164] The present application embodiment provides a terminal device, Figure 7 A schematic diagram of the structure of the terminal device provided in the embodiment of the present application is shown in FIG. Figure 7As shown, the terminal device 700 includes: a processor 701, a communication interface 702, a memory 703, and a communication bus for realizing data transmission between various parts. Among them, the communication interface 702 can assist the internal data transmission of the terminal device 700, and can also realize data transmission between external devices. For example, the communication interface 702, as an external communication interface, can realize data transmission between the terminal device 700 and output devices such as display screens. The external communication interface may include a standard wired interface and a wireless interface. Among them, the processor 701 is configured to execute the image processing program in the memory to implement the data recognition method provided in the above embodiment. It should be noted that the terminal device 700 may include but is not limited to the various components listed above, and the number of each component can be configured or adjusted in combination with actual needs and the situation of the terminal device 700 itself, and is not limited here.

[0165] The description of the above data identification device, terminal device and storage medium embodiments is similar to the description of the above method embodiments, and has similar technical descriptions and beneficial effects as the corresponding method embodiments. Due to space limitations, the above method embodiments can be used as examples, so they will not be repeated here. For technical details not disclosed in the data identification device, terminal device and storage medium embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0166] It should be understood that the "one embodiment" or "one embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence number of the embodiment of the present application is only for description and does not represent the advantages and disadvantages of the embodiment. It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the element.

[0167] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0169] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0170] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art can understand that all or part of the steps of the above method embodiments can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above method embodiments are executed; and the above storage medium includes: mobile storage devices, read-only memory (ROM), disks or optical disks, etc. Various media that can store program codes.

[0171] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art can be stored in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a terminal device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks. The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, and they should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data identification method, characterized in that: The method comprises: Performing scale recognition on the acquired target image to obtain attribute information of multiple scales; Performing pixel segmentation on the target image to respectively determine the pixel position of the pointer and the pixel positions of the multiple scales; Based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales, determining matching information between the attribute information of the plurality of scales and the pixel positions; Based on the pixel position of the pointer and the pixel positions of the plurality of scales, determining a target scale matching the pointer among the plurality of scales; Based on the matching information of the target scale and the pixel position of the pointer, data information in the target image is determined.

2. The method according to claim 1, characterized in that The step of performing scale recognition on the acquired target image to obtain attribute information of multiple scales includes: Performing scale value detection in the target image to obtain scale value positions of the multiple scales; Performing scale value identification on the scale at each scale value position to obtain scale values ​​of the multiple scales; The scale value positions of the multiple scales and the scale values ​​are determined as the attribute information.

3. The method according to claim 1 or 2, characterized in that: The pixel segmentation of the target image to respectively determine the pixel position of the pointer and the pixel positions of the plurality of scales includes: Determining a pixel level of the target image; The pointer and the multiple scales in the target image are segmented using the pixel level as a segmentation unit to obtain the pixel position of the pointer and the pixel positions of the multiple scales.

4. The method according to any one of claims 1 to 3, characterized in that: The determining, based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales, matching information between the attribute information of the plurality of scales and the pixel positions comprises: Determine a first distance between a scale value position in the attribute information of the plurality of scales and the pixel position; When the first distance is less than a preset distance threshold, the scale value position corresponding to the first distance, the pixel position and the scale value corresponding to the scale value position are associated to obtain matching information of the scale corresponding to the first distance.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining a target scale matching the pointer from among the multiple scales based on the pixel position of the pointer and the pixel positions of the multiple scales comprises: determining a second distance between a pixel position of the pointer and a pixel position of each of the plurality of scales; Based on the second distance, a scale adjacent to the pointer is determined as the target scale.

6. The method according to any one of claims 1 to 5, characterized in that: The step of determining data information in the target image based on the matching information of the target scale and the pixel position of the pointer includes: Determine angle information between the target scale and the pointer based on the pixel position of the target scale and the pixel position of the pointer in the matching information; Determining the indicated value of the pointer based on the angle information and the scale value of the target scale in the matching information; The indicated value of the pointer is determined as the data information in the target image.

7. The method according to claim 6, characterized in that The target scale includes two scales adjacent to the pointer, and determining the indicated value of the pointer based on the angle information and the scale value of the target scale in the matching information includes: Determining a ratio between the angle information of the two scales; Based on the ratio and the scale value of each of the two scales, the indicated value of the pointer is determined.

8. The method according to any one of claims 1 to 7, characterized in that: Before performing scale recognition on the acquired target image to obtain attribute information of multiple scales, the method further includes: The acquired picture includes the original image of the meter; Performing meter detection on the original image to obtain a detection frame of the meter; Performing image correction on the image within the detection frame to obtain the target image.

9. A data identification device, characterized in that: The device comprises: A first recognition module is used to perform scale recognition on the acquired target image to obtain attribute information of multiple scales; A first segmentation module, used for performing pixel segmentation on the target image to respectively determine the pixel position of the pointer and the pixel positions of the plurality of scales; A first determination module, configured to determine matching information between the attribute information of the plurality of scales and the pixel positions of the plurality of scales based on the attribute information of the plurality of scales and the pixel positions of the plurality of scales; A second determination module, configured to determine a target scale matching the pointer from among the multiple scales based on the pixel position of the pointer and the pixel positions of the multiple scales; The third determination module is used to determine the data information in the target image based on the matching information of the target scale and the pixel position of the pointer.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which, after being executed, can implement the data identification method described in any one of claims 1 to 8.

11. A terminal device, characterized in that: The terminal device includes a memory and a processor, the memory stores computer executable instructions, and the processor can implement the data identification method described in any one of claims 1 to 8 when running the computer executable instructions on the memory.

Images