Pointer instrument reading recognition method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a pointer instrument reading recognition method and device, electronic equipment and a storage medium, the method comprising: acquiring a to-be-tested image, and inputting the to-be-tested image into a deep learning network model to obtain feature data of a plurality of key points; wherein every three key points construct a circular arc line, and every two circular arc lines with different radii but the same center constitute an arc-shaped area containing a pointer instrument scale of the to-be-tested image; the matching degrees of the feature data of each key point on the to-be-tested image and the feature data of the key points on a template image corresponding to the to-be-tested image are calculated, and an arc-shaped area containing a pointer is obtained based on the matching degrees, and a pointer reading in the arc-shaped area is recognized. In this way, the feature data of the key points of the to-be-tested image can be directly recognized, and then the pointer reading can be accurately recognized, the accuracy of recognition is improved, and the training process of the model can be simplified, and the model can recognize the key points.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for recognizing pointer instrument readings. Background Technology

[0002] With the continuous improvement and development of science and technology, computer technology, deep learning and other technologies, the method of reading and recognizing instruments through image processing technology has been gradually used in practical applications. Pointer instruments have advantages such as simple structure, resistance to moisture, low cost, high accuracy, resistance to high temperature and electromagnetic interference. Therefore, pointer instruments are widely used in various fields such as machinery, medical, military, civil engineering, and chemical engineering.

[0003] In existing technologies, the reading recognition target image can be obtained by inputting the dial area image into a pre-trained meter detection model, and the meter reading can be obtained by further recognizing the target image.

[0004] However, the above methods require a large number of training samples to train the meter detection model, which consumes a lot of time. Moreover, the more types of meters that need to be detected, the more training data there is, and the more complex the meter detection model becomes, which reduces the accuracy of the detection. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for identifying pointer-type instrument readings, which can simplify the training process of instrument reading identification models, save time, and improve detection accuracy.

[0006] In a first aspect, this application provides a method for recognizing the readings of a pointer-type instrument, the method comprising:

[0007] The image to be tested is acquired and input into a deep learning network model to obtain feature data of multiple key points; wherein, every three key points construct a circular arc, and every two circular arcs with different radii but coincident centers constitute an arc-shaped area containing the pointer scale of the image to be tested.

[0008] Calculate the matching degree between the feature data of each key point on the image under test and the feature data of the key points on the template image corresponding to the image under test, and obtain an arc-shaped region containing a pointer based on the matching degree, and identify the pointer reading in the arc-shaped region.

[0009] Optionally, the feature data of the key points includes feature point data and descriptor data; the image to be tested is input into a deep learning network model to obtain feature data of multiple key points, including:

[0010] The image to be tested and the template image corresponding to the image to be tested are input into a deep learning network model for dimensionality reduction processing to obtain the first data corresponding to the image to be tested and the second data corresponding to the template image.

[0011] The first data and the second data are processed using a loss function to obtain multiple feature point data and multiple descriptor data corresponding to the image to be tested and the template image, respectively; the feature point data includes the position coordinates of key points; the descriptor data includes the direction of the key points, the corresponding radius, and the corresponding pixel.

[0012] Optionally, calculating the matching degree between the feature data of each key point in the image to be tested and the feature data of the key points in the template image corresponding to the image to be tested includes:

[0013] The feature data of each key point on the image to be tested is input into the noise reduction autoencoder to filter out the noise data in the feature data;

[0014] Calculate the matching degree between the feature data after filtering out noise data and the feature data corresponding to key points on the template image corresponding to the image to be tested.

[0015] Optionally, based on the matching degree, an arc-shaped region containing the pointer is obtained, and the pointer reading in the arc-shaped region is identified, including:

[0016] Based on the matching degree between the feature data of each key point on the image under test and the feature data of the key points on the template image corresponding to the image under test, the matching feature data corresponding to the image under test is searched sequentially.

[0017] The matching feature data is mapped onto the image to be tested to obtain an arc-shaped region containing the pointer, and the pointer position in the arc-shaped region is located using the vertical projection method of the image, and the corresponding pointer reading is identified.

[0018] Optionally, based on the matching degree between the feature data of each key point in the image to be tested and the feature data of the key points in the template image corresponding to the image to be tested, the matching feature data corresponding to the image to be tested is sequentially searched, including:

[0019] For each key point on the image to be tested, multiple matching degrees are obtained between the feature data of the key point and the feature data corresponding to the key point on the template image, and the target feature data is determined based on the maximum value among the multiple matching degrees.

[0020] The target feature data corresponding to each key point on the image under test and the feature data corresponding to the key points on the template image are input into the attention map neural network to obtain the first feature matrix corresponding to the image under test and the second feature matrix corresponding to the template image.

[0021] Calculate the matching score between the first feature matrix and the second feature matrix, and search for matching feature data sequentially based on the matching score.

[0022] Optionally, the matching score between the first feature matrix and the second feature matrix is ​​calculated, and matching feature data is sequentially searched based on the matching score, including:

[0023] Calculate the inner product of the first feature matrix and the second feature matrix to obtain N sets of matching scores, each set of matching scores including M matching scores; the first feature matrix has N rows and the second feature matrix has M columns;

[0024] Select the feature data corresponding to the maximum value in the N sets of matching scores in sequence, and filter the N feature data found based on the filtering channel to filter out the data corresponding to the key points that do not match the template image;

[0025] The filtered N feature data are normalized to obtain the optimal matching feature data.

[0026] Optionally, the matching feature data is mapped onto the image to be tested to obtain an arc-shaped region containing pointers, including:

[0027] Calculate the matching pair information of the matching feature data; the matching pair information is the position coordinates and matching score of the key point on the test image that matches the key point on the template image.

[0028] Based on the matching pair information, a homography matrix is ​​calculated, and the homography matrix is ​​used to project onto the corresponding template image;

[0029] The homography matrix is ​​subjected to an affine transformation with the coordinates of the key points on the template image to obtain an arc-shaped region containing pointers.

[0030] Optionally, the pointer position in the arc-shaped region is located using the vertical projection method of the image, and the corresponding pointer reading is identified, including:

[0031] The arc-shaped region is preprocessed, and the preprocessed arc-shaped region is then transformed into polar coordinates to obtain a rectangular region; the preprocessing is used to remove interference data.

[0032] The rectangular region is binarized, and the position coordinates of the corresponding projection pointer after binarization are obtained by the vertical projection method.

[0033] The pointer position is obtained based on the position coordinates, and the pointer reading corresponding to the pointer position is identified based on the range; the range is used to identify the pointer readings corresponding to different regions, and each range corresponds to a pointer reading.

[0034] Secondly, this application provides a pointer-type instrument reading device, the device comprising:

[0035] The acquisition module is used to acquire the image to be tested and input the image to be tested into the deep learning network model to obtain feature data of multiple key points; wherein, every three key points construct a circular arc, and every two circular arcs with different radii but coincident centers constitute an arc-shaped area containing the pointer scale of the image to be tested;

[0036] The processing module is used to calculate the matching degree between the feature data of each key point on the image under test and the feature data of the key points on the template image corresponding to the image under test, and to obtain an arc-shaped region containing a pointer based on the matching degree, and to identify the pointer reading in the arc-shaped region.

[0037] Thirdly, this application provides an electronic device comprising: a processor, a memory, and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program including instructions for performing the pointer-type instrument reading recognition method as described in any of the first aspects.

[0038] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the pointer-type instrument reading recognition method as described in any one of the first aspects.

[0039] In summary, this application provides a method, apparatus, electronic device, and storage medium for recognizing pointer instrument readings. It acquires a test image and inputs it into a deep learning network model to obtain feature data for multiple key points. Each set of three key points can construct a circular arc, and each pair of circular arcs with different radii but overlapping centers can form an arc-shaped region containing the pointer instrument scale of the test image. Furthermore, the matching degree between the feature data of each key point in the test image and the feature data of the corresponding key points in the template image is calculated. Based on the matching degree, the arc-shaped region containing the pointer is obtained, and the pointer reading within the arc-shaped region is recognized. This allows for direct recognition of the feature data of the key points in the test image, thereby accurately recognizing the pointer reading, improving recognition accuracy, and simplifying the model training process. It does not require a large number of training samples to train the model; only training the model to recognize key points is necessary. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0042] Figure 2 A flowchart illustrating a pointer-type instrument reading recognition method provided in an embodiment of this application;

[0043] Figure 3 A schematic diagram of the SuperPoint network architecture for image processing provided in an embodiment of this application;

[0044] Figure 4 A schematic diagram of the Denoising Autoencoder network processing flow provided in the embodiments of this application;

[0045] Figure 5 This is a schematic diagram of the SuperGlue network processing flow provided in the embodiments of this application;

[0046] Figure 6 A schematic diagram of a pointer positioning process provided in an embodiment of this application;

[0047] Figure 7 A flowchart illustrating a specific pointer-type instrument reading recognition method provided in this application embodiment;

[0048] Figure 8 This is a schematic diagram of the structure of a pointer-type instrument reading recognition device provided in an embodiment of this application;

[0049] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0050] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and purpose. For example, "first device" and "second device" are merely used to distinguish different devices and do not limit their order of execution. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0053] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0054] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0055] The embodiments of this application will now be described in conjunction with the accompanying drawings. Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. The pointer-type instrument reading recognition method provided in this application can be applied to, for example... Figure 1The application scenario shown includes: an analog meter 101, a camera 102, a server 103, a terminal device 104, and a user 105. In this scenario, the camera 102 can capture an image of the dial of the analog meter 101 and send the captured image to the server 103 for processing. Furthermore, the server 103 can receive the dial image as input into a deep learning network model and perform meter dial reading recognition. Finally, the server 103 can send the recognized result to the terminal device 104 with a display screen. The terminal device 104 receives the recognized result and displays it on the screen for the user 105 to view.

[0056] It is understood that the camera 102 can capture dial images of the pointer instrument 101 according to a preset cycle, or capture dial images of other pointer instruments. In this embodiment, the number of dial images input into the deep learning network model is not limited, but the dial images input into the model will be processed sequentially.

[0057] It should be noted that the terminal device 104 can be a large screen (or smart screen), mobile phone, tablet computer, smartwatch, smart bracelet, smart glasses or other terminal devices with a display screen, etc., and the embodiments of the present invention do not specifically limit it.

[0058] In existing technologies, meter readings are identified through manual recording. However, given the large number and variety of meters, and in harsh conditions with limited personnel, manual intervention has limitations. This is because pointer meters are diverse and their installation locations are complex, making manual meter reading extremely time-consuming and resource-intensive. Furthermore, manual meter reading is prone to fatigue, leading to missed or incorrect readings.

[0059] Therefore, a common approach is to input the dial area image into a pre-trained meter detection model to obtain the reading recognition target image, and then further recognize the target image to obtain the meter reading.

[0060] However, the pre-trained meter detection model in the above method requires a large number of training samples to be trained before it can be used, which consumes a lot of time. Moreover, the more types of meters that need to be detected, the more training data there is, and the more complex the meter detection model becomes, which reduces the accuracy of detection.

[0061] Therefore, this application provides a pointer instrument reading recognition method. By inputting the image to be tested into a deep learning network model, multiple key points can be extracted from the dial area of ​​the image to be tested to form an arc-shaped area containing the pointer scale of the image to be tested. Furthermore, it is matched with a template image to identify the pointer reading in the arc-shaped area. This effectively solves the problem of insufficient stability in the dial feature extraction process and greatly improves the accuracy of feature matching and the detection speed.

[0062] It is understood that the deep learning network model provided in this application only needs to use virtual 3D objects as the training dataset for the deep learning network model during the training process. It is sufficient to train the deep learning network model to have the function of extracting corner points. It does not require a large number of training samples containing dial images to train the model, thus saving training time.

[0063] The technical solutions of this application will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0064] For example, Figure 2 This is a flowchart illustrating a method for recognizing the readings of a pointer-type instrument, as provided in an embodiment of this application. Figure 2 As shown, the method may include:

[0065] S201. Obtain the image to be tested and input the image to be tested into a deep learning network model to obtain feature data of multiple key points; wherein, every three key points construct a circular arc, and every two circular arcs with different radii but coincident centers constitute an arc-shaped area containing the pointer scale of the image to be tested.

[0066] In this embodiment, deep learning refers to learning the inherent patterns and representation levels of sample data, enabling machines to have analytical and learning capabilities like humans, and to recognize data such as text and images. Specifically, deep learning network models can include Convolutional Neural Network (CNN) models, Dynamic Bayes Network (DBN) models, and stacked auto-encoder network models, which can be used to automatically extract features from data. This embodiment does not limit the specific model corresponding to the deep learning network model.

[0067] In this step, the key points can be the starting position, ending position, and center position of the dial scale in the image to be tested, as well as the starting position, ending position, and center position of the dial scale after the radius of the image to be tested is reduced to a predefined size. The predefined size can be determined according to different dial features, so that the circular arc formed by the starting position, ending position, and center position of the dial after the radius is reduced is below the dial scale and intersects with all possible ranges of pointer movement. For example, the predefined size is one-tenth of the radius of the dial of the image to be tested.

[0068] Specifically, the process of inputting the image to be tested into a deep learning network model to obtain multiple key points can be as follows: Using the center of the dial of the image to be tested as the center point, the distance from the starting point of the scale to the center point as the radius, and the starting position of the scale as the starting point and the ending position of the scale as the ending point, draw an arc, i.e., a circular arc constructed by three key points. Select the starting point, midpoint, and ending point of the arc as the three key points to be labeled. Further, after reducing the radius of the dial of the image to be tested by one-tenth, draw another arc using the starting position of the scale as the starting point and the ending position of the scale as the ending point. Select the starting point, midpoint, and ending point of the corresponding radius arc as the other three key points to be labeled. The above two circular arcs with different radii but overlapping centers can constitute an arc-shaped area containing the pointer scale of the image to be tested.

[0069] Understandably, when training a deep learning network model, the above process can be used to annotate the six key points in the dial image. Furthermore, the dial image and the corresponding six key points can be used as training data and input into the deep learning network model for training. The training data can be different types of dial images and their corresponding six key points, with multiple training data constituting the training dataset. Data augmentation operations can also be used to randomly rotate and scale each dial image to create new images, enriching the training dataset and thereby training the deep learning network model to extract corner points.

[0070] It should be noted that the number of key points obtained by inputting the image to be tested into the deep learning network model in this application embodiment is not specifically limited. For example, it can also be 4 key points on one circular arc, and 3 or 4 key points on another circular arc. This application does not make specific limitations on this. The above is just an example. Three points are the minimum number of points to form a circular arc. Other numbers of key points are also possible.

[0071] In this embodiment of the application, the feature data of the key point includes feature point data and descriptor data. The feature point data is used to characterize the positional feature information of the key point and may include data such as the position coordinates of the key point. The descriptor data is used to characterize the data structure feature information of the key point and may include data such as the direction of the key point, the corresponding radius, and the corresponding pixel.

[0072] For example, in Figure 1 In the application scenario, server 103 can receive the dial image of pointer meter 101 sent by camera 102. Furthermore, by inputting the dial image of pointer meter 101 into a deep learning network model, feature data of six key points can be obtained, such as the feature data of the starting point, middle point, and ending point of the dial image of pointer meter 101, and the feature data of the starting point, middle point, and ending point of the dial image of pointer meter 101 after the radius of the dial image of pointer meter 101 is reduced by one-tenth.

[0073] S202. Calculate the matching degree between the feature data of each key point on the image to be tested and the feature data of the key points on the template image corresponding to the image to be tested, and obtain an arc-shaped region containing a pointer based on the matching degree, and identify the pointer reading in the arc-shaped region.

[0074] In this step, the template image can refer to the image of the dial type corresponding to the image under test stored in the deep learning network model. The trained deep learning network model ensures that any image under test has a corresponding template image. After obtaining the template image corresponding to the image under test, it is possible to know which key points' feature data to extract from the image under test. The template image can be manually input or can be identified by inputting the image under test into the deep learning network model. This application embodiment does not specifically limit this.

[0075] In this embodiment of the application, the matching degree can refer to the data-based measurement of the similarity between the feature data of each key point on the image to be tested and the feature data of the key point on the template image corresponding to the image to be tested. By finding the feature data pair with the highest matching degree, the feature data of the corresponding optimal key point on the image to be tested can be determined. Here, the feature data pair refers to the feature data of a key point on the image to be tested and the feature data of a key point on the template image corresponding to the image to be tested.

[0076] It is understandable that by inputting the image to be tested into a deep learning network model, multiple sets of feature data for key points are obtained. By calculating the similarity between the feature data of each key point on the image to be tested and the feature data of the key points on the template image corresponding to the image to be tested in each set of data, the optimal feature data of the key points can be found, and thus the arc-shaped region containing the pointer can be determined.

[0077] For example, in Figure 1In the application scenario, after the server 102 obtains the feature data of multiple sets of key points in the dial image of the pointer instrument 101, the feature data of each set of key points includes the feature data of six key points. Further, the matching degree of the feature data of each key point on the dial image of the pointer instrument 101 with the feature data of the key points on the template image corresponding to the dial image of the pointer instrument 101 can be calculated. Further, the feature data pair with the highest matching degree is found, and then the feature data of the optimal six key points on the image to be tested is determined. Then, the feature data of the optimal six key points can be mapped to form an arc-shaped area containing the pointer. In this way, the pointer reading in the arc-shaped area can be identified.

[0078] Therefore, this application provides a method for recognizing pointer instrument readings. This method involves acquiring a test image and inputting it into a deep learning network model to obtain feature data of multiple key points on two circular arcs. The two arcs have different radii but coincident centers, forming an arc-shaped region containing the pointer scale of the test image. Further, by calculating the matching degree between the feature data of each key point on the test image and the feature data of the corresponding key points on the template image, the optimal key point's feature data is determined based on the matching degree of each feature data pair. This yields the arc-shaped region containing the pointer, allowing for the recognition of the pointer reading within the arc-shaped region. This method directly identifies the feature data of the key points in the test image, thereby accurately recognizing the pointer reading, improving recognition accuracy, and simplifying the model training process. It does not require a large number of training samples to train the model; only training the model to recognize key points is necessary.

[0079] Optionally, the feature data of the key points includes feature point data and descriptor data; the image to be tested is input into a deep learning network model to obtain feature data of multiple key points, including:

[0080] The image to be tested and the template image corresponding to the image to be tested are input into a deep learning network model for dimensionality reduction processing to obtain the first data corresponding to the image to be tested and the second data corresponding to the template image.

[0081] The first data and the second data are processed using a loss function to obtain multiple feature point data and multiple descriptor data corresponding to the image to be tested and the template image, respectively; the feature point data includes the position coordinates of key points; the descriptor data includes the direction of the key points, the corresponding radius, and the corresponding pixel.

[0082] In this step, the first data can refer to the data after the size of the image to be tested has been reduced through convolutional layer processing, and the second data can refer to the data after the size of the template image has been reduced through convolutional layer processing.

[0083] In one possible implementation, a test image and a template image corresponding to the manually input test image are obtained, and the test image and the template image are input into a deep learning network model to obtain feature data of multiple key points corresponding to the test image and feature data of multiple key points corresponding to the template image.

[0084] In another possible implementation, the image to be tested is acquired and input into a deep learning network model, which can directly obtain the template image corresponding to the image to be tested. Furthermore, the deep learning network model processes the image to be tested and the template image to obtain feature data of multiple key points corresponding to the image to be tested and feature data of multiple key points corresponding to the template image.

[0085] Specifically, the process of inputting the test image and template image into a deep learning network model to obtain feature point data and descriptor data can be understood as the process of SuperPoint network (i.e., fully convolutional neural network structure) performing feature extraction to obtain feature points and descriptors, mainly including the following four parts: encoding network (Shared Encoder), decoding network (Interest Point Decoder), descriptor detection (Descriptor Decoder), and loss function construction. The process of SuperPoint network processing the test image and template image is similar, and will not be described again here. The following explanation uses SuperPoint network processing the test image as an example.

[0086] For example, Figure 3 This is a schematic diagram of the SuperPoint network architecture for image processing provided in an embodiment of this application, as shown below. Figure 3 As shown, the steps of SuperPoint network in processing images are as follows:

[0087] Step 1: Input the image to be tested (size H×W) into the SuperPoint network. Dimensionality reduction is performed through a shared encoding network. Specifically, after the image is input into the SuperPoint network, a VGG-style encoder is used to reduce the image size. The encoder includes convolutional layers, max-pooling layers, and non-linear activation layers. Further, three max-pooling layers transform the image size from H×W to H. C =H / 8 and W C =W / 8, after passing through the encoder, the image is composed of I∈R H×W Transform into a tensor Where H is the original image size of the image to be tested, HC After the dimensionality reduction, proceed with steps 2 and 3, which are executed simultaneously.

[0088] Step 2: The dimensionality-reduced image is processed by a feature point decoder to obtain the test image containing feature points. Specifically, sub-pixel convolution is used to reduce computational cost and thus decrease model computation (sub-pixel convolution). For example, the dimension of the input tensor (i.e., the image size) is... The output dimension is R. H×W 65 represents the 8x8 local region of the original image. Furthermore, by optimizing the feature point positions using the normalization exponent (Softmax) function and processing with the Reshape function, more accurate positional information can be obtained.

[0089] Step 3: The dimensionality-reduced image is processed by a descriptor decoder to obtain the test image containing descriptors, i.e., descriptor detection. Specifically, firstly, a semi-dense descriptor is obtained using a network similar to UCN, which reduces the algorithm's training memory overhead and running time. Further, the remaining descriptors are obtained through bicubic interpolation, and then the descriptors are normalized using the L2 norm (Euclidean distance) to obtain a uniform length description. At this point, the feature dimension is reduced from... Change to R H×W×D .

[0090] Since the network is divided into two branches, the loss function consists of two terms, one of which is the feature point. Another item is the descriptor. The coefficient λ represents the weight corresponding to the descriptor; therefore, the loss function can be expressed by the following formula:

[0091]

[0092] Furthermore, by using the aforementioned loss function to process the test image containing feature points and the test image containing descriptors, we can obtain the feature points and descriptors of the test image, namely feature point data and descriptor data. The feature point data corresponds to the position coordinates of the key points, etc.; the descriptor data corresponds to the direction, radius, and pixel number of the key points, etc.

[0093] Therefore, the deep learning network model in this embodiment can be combined with the trained SuperPoint network to extract feature point data and descriptive operator data, which greatly improves the accuracy of data extraction.

[0094] Optionally, calculating the matching degree between the feature data of each key point in the image to be tested and the feature data of the key points in the template image corresponding to the image to be tested includes:

[0095] The feature data of each key point on the image to be tested is input into the noise reduction autoencoder to filter out the noise data in the feature data;

[0096] Calculate the matching degree between the feature data after filtering out noise data and the feature data corresponding to key points on the template image corresponding to the image to be tested.

[0097] In this step, the feature data of multiple key points, namely the feature point data and descriptor data corresponding to multiple key points, can be processed by a denoising autoencoder. The input feature data is compressed, encoded and then decoded to filter out noise data and extract the most representative information from the data, thus obtaining updated feature point data and descriptor data.

[0098] Denoising Autoencoder, building upon the autoencoder, introduces noise into the input data (the network's input layer) to prevent overfitting. This makes the trained encoder more robust, thereby enhancing the generalization ability of the deep learning network model. For example, some pixels can be randomly changed from 0 to 1 and then fed into the deep learning network model, with the expectation that it can reconstruct the original image without noise.

[0099] Specifically, Figure 4 This is a schematic diagram of the Denoising Autoencoder network processing flow provided in the embodiments of this application, as shown below. Figure 4 As shown, taking the feature data of a keypoint as an example, where x is the original feature data of the keypoint, the Denoising Autoencoder sets the value of the input layer node to 0 with a certain probability, thereby obtaining the model input after filtering out noise data. Furthermore, the feature data is processed using existing autoencoder procedures, i.e., it passes through an intermediate layer f. θ The data y is obtained through processing, and then the data y is processed by the decoding layer g. θ' Decode the data and output it to the output layer of n neurons to obtain data z. If the input data is a bit vector or a bit probability vector, the cross-entropy method can be used to process x. The formula for the cross-entropy method is as follows:

[0100] For example, in Figure 1In the application scenario, server 102 can input the feature data of each key point in the dial image of pointer instrument 101 into the noise reduction auto encoder to filter out noise data in the feature data. Since server 102 can obtain feature data of multiple sets of key points, and each set of key point feature data includes feature data of six key points, further, the matching degree of the feature data after filtering out noise data and the feature data corresponding to the key points on the template image corresponding to the dial image of pointer instrument 101 can be calculated respectively. Each set can obtain 6 matching degrees.

[0101] Therefore, the deep learning network model in this application embodiment can be combined with a noise reduction autoencoder to filter out noisy data, thereby improving the robustness of data processing.

[0102] Optionally, based on the matching degree, an arc-shaped region containing the pointer is obtained, and the pointer reading in the arc-shaped region is identified, including:

[0103] Based on the matching degree between the feature data of each key point on the image under test and the feature data of the key points on the template image corresponding to the image under test, the matching feature data corresponding to the image under test is searched sequentially.

[0104] The matching feature data is mapped onto the image to be tested to obtain an arc-shaped region containing the pointer, and the pointer position in the arc-shaped region is located using the vertical projection method of the image, and the corresponding pointer reading is identified.

[0105] In this embodiment of the application, the vertical projection method of the image can refer to the geometric projection method of projecting the ground point along the vertical line onto the horizontal plane to obtain the planar position of the ground point on the horizontal plane, which constitutes the corresponding planar figure of the ground point and is used to locate the pointer position in the image.

[0106] In this step, a feature matching algorithm based on graph convolutional neural networks (SuperGlue) can be used for feature matching. That is, by matching the feature data of each key point in the image to be tested with the matching degree of the key points in the corresponding template image, the matching feature data corresponding to the image to be tested is found sequentially. Since there are multiple sets of feature data for each key point, there are multiple matching degrees between each key point feature data and the key points in the corresponding template image. Further, the matching feature data is mapped to the image to be tested. That is, by performing an affine transformation with the coordinates marked on the template image, the six key points can be mapped to the image to be tested. The arc-shaped area containing the pointer can be obtained and extracted. Then, the pointer position is located using the vertical projection method of the image, and finally the pointer reading can be identified.

[0107] Therefore, the deep learning network model in this embodiment can be combined with SuperGlue to obtain matching information, which increases the accuracy of feature matching. It can also be combined with the vertical projection method of the image to locate the pointer position, so that the pointer area can be extracted and located under complex background patterns.

[0108] Optionally, based on the matching degree between the feature data of each key point in the image to be tested and the feature data of the key points in the template image corresponding to the image to be tested, the matching feature data corresponding to the image to be tested is sequentially searched, including:

[0109] For each key point on the image to be tested, multiple matching degrees are obtained between the feature data of the key point and the feature data corresponding to the key point on the template image, and the target feature data is determined based on the maximum value among the multiple matching degrees.

[0110] The target feature data corresponding to each key point on the image under test and the feature data corresponding to the key points on the template image are input into the attention map neural network to obtain the first feature matrix corresponding to the image under test and the second feature matrix corresponding to the template image.

[0111] Calculate the matching score between the first feature matrix and the second feature matrix, and search for matching feature data sequentially based on the matching score.

[0112] In this step, since there are multiple sets of feature data for each key point on the image to be tested, such as when the image to be tested is input into a deep learning network model, it can correspond to images with multiple rotation angles. Therefore, there are multiple matching degrees between each key point feature data and the key points on the corresponding template image. By finding the maximum value among the multiple matching degrees calculated between each key point and the key points on the template image, the target feature data can be determined, that is, the key point data on the image to be tested that best matches the key points on the corresponding template image.

[0113] In this embodiment, the attention graph neural network refers to a network that uses an attention mechanism to aggregate neighboring nodes, thereby adaptively assigning weights to different neighbors and improving the expressive power of the graph neural network model. The attention mechanism processes the location and visual appearance of feature points, which can be understood as humans browsing two images back and forth to tentatively select matching key points and checking them repeatedly (if a feature doesn't match, they observe whether there are better matching points nearby, until a matching point is found / or no match is found). The attention mechanism can increase the specificity of feature points by actively seeking context, thus eliminating some singular matches and improving its specificity.

[0114] In this process, the target feature data and the feature data corresponding to key points on the template image are input into the attention map neural network to obtain the first feature matrix corresponding to the image under test and the second feature matrix corresponding to the template image. The purpose is to improve the robustness of the feature data corresponding to the image under test by summarizing the features of the key points themselves and the features of their neighbors to generate feature data that meets the requirements.

[0115] Furthermore, by calculating the matching scores corresponding to the first feature matrix and the second feature matrix, and sequentially searching for matching feature data based on the matching scores, i.e. the optimal feature data, the optimal feature data of multiple key points corresponding to the final image under test can be determined. The feature data of the optimal multiple key points can be used to construct an arc-shaped area containing pointer instrument scale.

[0116] Therefore, the embodiments of this application can select feature data with high matching degree with key points in the template image and calculate the matching score, thereby finding the optimal data and improving the accuracy of data acquisition.

[0117] Optionally, the matching score between the first feature matrix and the second feature matrix is ​​calculated, and matching feature data is sequentially searched based on the matching score, including:

[0118] Calculate the inner product of the first feature matrix and the second feature matrix to obtain N sets of matching scores, each set of matching scores including M matching scores; the first feature matrix has N rows and the second feature matrix has M columns;

[0119] Select the feature data corresponding to the maximum value in the N sets of matching scores in sequence, and filter the N feature data found based on the filtering channel to filter out the data corresponding to the key points that do not match the template image;

[0120] The filtered N feature data are normalized to obtain the optimal matching feature data.

[0121] In this step, the inner product of the first feature matrix and the second feature matrix is ​​calculated. That is, the first feature matrix is ​​divided into N row vectors and the second feature matrix is ​​divided into M column vectors. Thus, the image to be tested and the template image have M and N features, respectively. Further, the inner product of each row vector with the corresponding M column vectors is calculated to obtain M matching scores. The feature data of the key point corresponding to the maximum value among the M matching scores is found. This process is repeated for each of the N row vectors to obtain N sets of M matching scores. In this embodiment, the specific values ​​of M and N are not limited.

[0122] In this embodiment, a filtering channel (dustbin channel) is added to the last column / row of the calculated score matrix S (first feature matrix and second feature matrix). The purpose of this is to filter out incorrect matching points, such as blank or extra feature data (covered by shadows or occlusions), that is, the feature data of the key point does not actually have corresponding feature point data on the template image.

[0123] In conjunction with the above embodiments, Figure 5 This is a schematic diagram of the SuperGlue network processing flow provided in the embodiments of this application, as shown below. Figure 5 As shown, the SuperGlue network processing flow is as follows: Step 1: For the test image A and the template image B, each image has feature point positions p and corresponding descriptors d. The test image A and the template image B have M and N features respectively, which can be represented as... as well as The local features of the test image A and the template image B are input into the attention map neural network to obtain the first feature matrix corresponding to the test image A. The second feature matrix corresponding to template image B

[0124] Specifically, the feature point locations of the image A to be tested... and the feature point positions of template image B The visual descriptor of the image to be tested, A, is input into the keypoint encoder. Visual descriptor of template image B The input is fed into the attention aggregation mechanism, which makes the feature point locations and descriptors more specific for feature matching. Therefore, the feature point locations and descriptors are merged into the initial representation of each feature point ii. Here, MLP stands for Multilayer Perceptron (MLP), which is used to increase the dimensionality of low-dimensional features. The above formula actually couples the visual appearance with the position of feature points. Because of this, the subsequent attention mechanism can fully take into account the appearance and positional similarity of features.

[0125] The attention aggregation mechanism includes self-attention and cross-attention. When considering a single complete graph, its nodes are each feature point in the image. This graph includes two different types of undirected edges: one is "Intra-image edges" (self edge), which connects feature points from within the image; the other is "Inter-image edges" (cross edge), which connects feature point ii in this graph with all feature points in another graph, thus forming the edge. Here, self- / cross-attention actually simulates the process of humans browsing back and forth for matching. Self-attention is to make features more specific for matching, while cross-attention is to use these specific points to compare the similarity of features between images.

[0126] The attention mechanism is used for feature matching, which aggregates self and cross information to obtain m. ε→i The self-edge mechanism utilizes self-attention, while the cross-edge mechanism utilizes cross-attention. Similar to database retrieval, if you want to query q... i Based on the element's attribute, i.e., key k i The values ​​of certain elements can be retrieved. Right now Wherein, attention weight α ij It is the Softmax function that queries and retrieves the similarity between object key values, i.e.

[0127] Step 2: Based on the first feature matrix Second characteristic matrix Construct an assignment matrix P, which is a matrix consisting of matching feature data corresponding to each key, where P1 N ≤1 M and P T 1 M ≤1 N This assignment matrix can be calculated by computeding a score matrix S∈R. M×N This is achieved by maximizing the overall score ∑ i，j S ij P i，j We obtain this allocation matrix P, and it is important to note that P is constrained.

[0128] Specifically, it is calculated using the following formula. as well as The inner product is used to obtain the score:

[0129]

[0130] This yields N sets of M matching scores. The maximum value among these N sets of M matching scores is then determined to identify N feature data. Furthermore, based on the principle that feature points on image A are assigned to a feature match on image B or to a dustbin channel, this means that each dustbin has N and M matches, resulting in the following constraints on the assignment matrix:

[0131] and

[0132] in, The filtered N feature data can form an allocation matrix.

[0133] Step 3: Normalize the filtered N feature data, i.e., solve for maximizing the overall score. This can be solved using the Sinkhorn algorithm, which transforms one probability distribution into another with minimal cost. Specifically, for the assignment matrix, first perform row-by-row normalization by dividing each element of the first row by the sum of the elements in the first row to obtain a new "first row"; repeat this operation for each row; then perform column-by-column normalization. Repeat the above two steps, and perform partial assignments, until the result converges to a matrix composed of matching feature data where the sum of each row and column is also 1.

[0134] Understandably, the SuperGlue network processing workflow utilizes the Loss function, meaning that both the graph neural network and the calculation of the matching degree are differentiable. This makes backpropagation training possible, and the network training employs a supervised learning approach, where ground truth matching values ​​are available. (For example, matching relationships obtained from the true value relative pose transformation), and of course, some feature points that are not matched can also be obtained. as well as Given truth labels, we can minimize the assignment matrix. The negative log-likelihood function is used, therefore the original loss function is improved by balancing the loss term by assigning higher weights to positive pairs and by modifying the bin score value to balance accuracy and matching score. The loss function used is:

[0135]

[0136] In this way, the goal of supervised learning can simultaneously maximize both precision and recall in matching.

[0137] Therefore, this application embodiment combines with the SuperGlue network to match feature points, adds filtering channels to filter out shadows and occluded parts in the image, and improves the accuracy of data acquisition.

[0138] Optionally, the matching feature data is mapped onto the image to be tested to obtain an arc-shaped region containing pointers, including:

[0139] Calculate the matching pair information of the matching feature data; the matching pair information is the position coordinates and matching score of the key point on the test image that matches the key point on the template image.

[0140] Based on the matching pair information, a homography matrix is ​​calculated, and the homography matrix is ​​used to project onto the corresponding template image;

[0141] The homography matrix is ​​subjected to an affine transformation with the coordinates of the key points on the template image to obtain an arc-shaped region containing pointers.

[0142] In this step, the matching pair information of the matching feature data is calculated. That is, the LO-RANSAC algorithm can be used to purify the obtained matching feature data, filter out the wrong matching pairs, and then obtain the location coordinates and matching scores of the key points on the test image that match the key points on the template image.

[0143] Furthermore, based on the calculated matching pair information, the corresponding homography matrix H can be calculated. This homography matrix is ​​used to project onto the corresponding template image; that is, an affine transformation can be performed between the homography matrix and the coordinates marked on the template image. The homography matrix H can be expressed as:

[0144] Here, f(x,y) is the function corresponding to the image.

[0145] A homography matrix H is essentially a 3×3 perspective transformation matrix. This matrix projects points in an image onto their corresponding image pairs. The corresponding formula is:

[0146]

[0147] Specifically, taking the feature data of 6 key points as an example, the server can map the feature data of the 6 key points onto the image under test. Connecting the 6 points can obtain an arc-shaped region along the scale direction. Furthermore, an arc-shaped region containing the pointer can be extracted. (See attached diagram.) Figure 6 Figure B in the diagram.

[0148] Therefore, the embodiments of this application can map the key point information onto the image to obtain an arc-shaped area containing the pointer, which can be recognized and improve the recognition accuracy.

[0149] Optionally, the pointer position in the arc-shaped region is located using the vertical projection method of the image, and the corresponding pointer reading is identified, including:

[0150] The arc-shaped region is preprocessed, and the preprocessed arc-shaped region is then transformed into polar coordinates to obtain a rectangular region; the preprocessing is used to remove interference data.

[0151] The rectangular region is binarized, and the position coordinates of the corresponding projection pointer after binarization are obtained by the vertical projection method.

[0152] The pointer position is obtained based on the position coordinates, and the pointer reading corresponding to the pointer position is identified based on the range; the range is used to identify the pointer readings corresponding to different regions, and each range corresponds to a pointer reading.

[0153] In this embodiment, preprocessing refers to grayscale processing of the image, while binarization processing refers to changing the pixels in the image to make the entire image exhibit a clear black and white effect, as shown in the attached figure. Figure 6 Figure C in the diagram.

[0154] In this step, the range can refer to the pointer reading corresponding to a certain area of ​​the pointer in a pre-set manner. The range can be stored in a range lookup table, in which each range corresponds to a pointer reading. This application embodiment does not specifically limit the storage location of the range and its corresponding pointer reading.

[0155] For example, image processing techniques can be used to first preprocess the captured arc-shaped region to remove some interference information. Then, polar coordinate transformation can be used to convert the arc-shaped region into a rectangular region. Specifically, the arc-shaped region can be transformed into a rectangular region using the conversion formula between the polar coordinate system and the Cartesian coordinate system. The corresponding formula is:

[0156]

[0157] Where M(ρ, θ) are the polar coordinates of point M, M′(x, y) are the rectangular coordinates of the corresponding point M, and M is the key point.

[0158] Furthermore, by locating the pointer's position using the vertical projection method and combining this location with the position of the pointer within the rectangular frame and the measurement range, the final instrument reading can be obtained. Specifically, the rectangular area is binarized, and a binary image of each column is obtained through vertical projection. The pointer position is determined based on the projected coordinates, and finally, the measurement range is combined to obtain the final instrument reading.

[0159] For example, Figure 6A schematic diagram of a pointer positioning process provided in an embodiment of this application is shown below. Figure 6 As shown, the test image A of the pointer dial is obtained and input into a deep learning network model for processing to obtain an arc-shaped region B containing the pointer. Further, the arc-shaped region is preprocessed to obtain an arc-shaped region C. Further, the arc-shaped region is transformed into polar coordinates and binarized to obtain a rectangular region D. Further, the position map E of the projected pointer corresponding to the rectangular region D can be obtained by using the vertical projection method, and then the pointer reading corresponding to the pointer position can be identified.

[0160] Therefore, the embodiments of this application can preprocess and binarize the arc-shaped region to make the pointer image clearer and improve the accuracy of recognition.

[0161] In conjunction with the above embodiments, Figure 7 A flowchart illustrating a specific pointer-type instrument reading recognition method provided in this application embodiment is shown below. Figure 7 As shown, the pointer-type instrument reading recognition method includes the following steps:

[0162] Step A: Before using the deep learning network model, the deep learning network model is pre-trained. That is, when using the deep learning network model, the key points of the test image and template image can be obtained and the arc-shaped region constructed by the key points can be obtained. Further, the test image and template image are obtained and the network is initialized. The test image and template image are input into the SuperPoint (i.e., deep learning network) model to obtain feature points and descriptors. Further, the feature points and descriptors are optimized by a noise reduction autoencoder to obtain multiple updated feature points and descriptors. Then, step B is executed.

[0163] Step B: Input the updated feature points and descriptor into the SuperGlue network for feature matching, and use the LO-RANSAC algorithm to eliminate incorrect matching pairs to obtain matching pair information, then proceed to step C.

[0164] Step C: Calculate the homography matrix based on the matching pair information, and map the homography matrix onto the image to be measured using the labeled coordinates to obtain the arc-shaped region. Further, transform the polar coordinates of the arc-shaped region into a rectangular region, and use the gradient projection method to process the rectangular region to locate the pointer region. Then, identify the pointer reading based on the measurement range, and finally output the result.

[0165] Based on this, this application proposes a deep learning-based pointer instrument reading recognition method. This method can recognize meter readings in real time. Compared with existing technologies, this application effectively solves the problems of insufficient stability and susceptibility to lighting in the dial feature extraction process by using a combination of SuperPoint, Denoising AutoEncoder, SuperGlue, and LO-RANSAC techniques in the dial area extraction. Compared with existing methods, it increases the accuracy of feature matching and has a significant speed advantage. SuperGlue is used for pointer position positioning. After obtaining the matching pair information, the homography matrix is ​​calculated. The homography matrix and the coordinates of the arc-shaped area marked in the template image are mapped to the position of the image to be tested, and then the arc-shaped area is extracted. The straightened rectangular area is obtained through polar coordinate transformation. Finally, the rectangular area is located by the vertical projection method of the image. This method can extract and locate the pointer area under complex background patterns.

[0166] In the foregoing embodiments, the pointer-type instrument reading method provided by the embodiments of this application has been described. To implement the functions of the methods provided by the embodiments of this application, the electronic device serving as the execution subject may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0167] For example, Figure 8 This is a schematic diagram of the structure of a pointer-type instrument reading device provided in an embodiment of this application, as shown below. Figure 8 As shown, the device includes an acquisition module 810 and a processing module 820. The acquisition module 810 is used to acquire the image to be tested and input the image to be tested into a deep learning network model to obtain feature data of multiple key points. Each set of three key points forms a circular arc, and each set of two circular arcs with different radii but coincident centers constitutes an arc-shaped area containing the pointer scale of the image to be tested.

[0168] The processing module 820 is used to calculate the matching degree between the feature data of each key point on the image under test and the feature data of the key points on the template image corresponding to the image under test, and to obtain an arc-shaped region containing a pointer based on the matching degree, and to identify the pointer reading in the arc-shaped region.

[0169] Optionally, the feature data of the key points includes feature point data and descriptor data; the acquisition module 810 is specifically used for:

[0170] The image to be tested and the template image corresponding to the image to be tested are input into a deep learning network model for dimensionality reduction processing to obtain the first data corresponding to the image to be tested and the second data corresponding to the template image.

[0171] The first data and the second data are processed using a loss function to obtain multiple feature point data and multiple descriptor data corresponding to the image to be tested and the template image, respectively; the feature point data includes the position coordinates of key points; the descriptor data includes the direction of the key points, the corresponding radius, and the corresponding pixel.

[0172] Optionally, the processing module 820 includes a calculation module and a recognition module, wherein the calculation module is used for:

[0173] The feature data of each key point on the image to be tested is input into the noise reduction autoencoder to filter out the noise data in the feature data;

[0174] Calculate the matching degree between the feature data after filtering out noise data and the feature data corresponding to key points on the template image corresponding to the image to be tested.

[0175] Optionally, the identification module includes a search unit and an identification unit;

[0176] Specifically, the search unit is used to sequentially search for matching feature data corresponding to the image under test based on the matching degree between the feature data of each key point on the image under test and the feature data corresponding to the key points on the template image corresponding to the image under test.

[0177] The recognition unit is used to map the matching feature data onto the image to be tested to obtain an arc-shaped region containing the pointer, and to locate the pointer position in the arc-shaped region using the vertical projection method of the image, and to identify the corresponding pointer reading.

[0178] Optionally, the search unit includes a determining unit, an input unit, and a calculation unit;

[0179] Specifically, the determining unit is used to obtain multiple matching degrees between the feature data of the key point and the feature data corresponding to the key point in the template image for each key point in the image to be tested, and to determine the target feature data based on the maximum value among the multiple matching degrees.

[0180] The input unit is used to input the target feature data corresponding to each key point on the image to be tested and the feature data corresponding to the key points on the template image into the attention map neural network to obtain the first feature matrix corresponding to the image to be tested and the second feature matrix corresponding to the template image.

[0181] The calculation unit is used to calculate the matching score between the first feature matrix and the second feature matrix, and to sequentially search for matching feature data based on the matching score.

[0182] Optionally, the computing unit is specifically used for:

[0183] Calculate the inner product of the first feature matrix and the second feature matrix to obtain N sets of matching scores, each set of matching scores including M matching scores; the first feature matrix has N rows and the second feature matrix has M columns;

[0184] Select the feature data corresponding to the maximum value in the N sets of matching scores in sequence, and filter the N feature data found based on the filtering channel to filter out the data corresponding to the key points that do not match the template image;

[0185] The filtered N feature data are normalized to obtain the optimal matching feature data.

[0186] Optionally, the identification unit is specifically used for:

[0187] Calculate the matching pair information of the matching feature data; the matching pair information is the position coordinates and matching score of the key point on the test image that matches the key point on the template image.

[0188] Based on the matching pair information, a homography matrix is ​​calculated, and the homography matrix is ​​used to project onto the corresponding template image;

[0189] The homography matrix is ​​subjected to an affine transformation with the coordinates of the key points on the template image to obtain an arc-shaped region containing pointers.

[0190] Optionally, the identification unit is specifically used for:

[0191] The arc-shaped region is preprocessed, and the preprocessed arc-shaped region is then transformed into polar coordinates to obtain a rectangular region; the preprocessing is used to remove interference data.

[0192] The rectangular region is binarized, and the position coordinates of the corresponding projection pointer after binarization are obtained by the vertical projection method.

[0193] The pointer position is obtained based on the position coordinates, and the pointer reading corresponding to the pointer position is identified based on the range; the range is used to identify the pointer readings corresponding to different regions, and each range corresponds to a pointer reading.

[0194] The specific implementation principle and effects of the pointer-type instrument reading device provided in this application embodiment can be found in the relevant descriptions and effects of the above embodiments, and will not be elaborated further here.

[0195] This application also provides a schematic diagram of the structure of an electronic device. Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device may include: a processor 901 and a memory 902 communicatively connected to the processor; the memory 902 stores a computer program; the processor 901 executes the computer program stored in the memory 902, causing the processor 901 to perform the method described in any of the above embodiments.

[0196] The memory 902 and the processor 901 can be connected via bus 903.

[0197] This application also provides a computer-readable storage medium storing computer program execution instructions, which, when executed by a processor, are used to implement the methods described in any of the foregoing embodiments of this application.

[0198] This application also provides a chip for executing instructions, which is used to perform the methods described in any of the foregoing embodiments executed by an electronic device as described in any of the foregoing embodiments of this application.

[0199] This application also provides a computer program product, which includes a computer program that, when executed by a processor, can implement the methods described in any of the foregoing embodiments executed by an electronic device as described in any of the foregoing embodiments of this application.

[0200] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0201] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0202] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0203] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0204] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0205] The memory may include high-speed random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0206] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0207] The aforementioned storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0208] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0209] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A method of reading a dial gauge, characterized in that, The method comprises: acquiring a to-be-tested image, inputting the to-be-tested image into a deep learning network model to obtain feature data of a plurality of key points; the plurality of key points are divided into a plurality of groups, and each group of key points corresponds to an image at a rotation angle; each group of key points comprises three first key points and three second key points, the three first key points construct a first circular arc, the three second key points construct a second circular arc, the first circular arc and the second circular arc have different radii but have the same center, and the two circular arcs form an arc-shaped area containing a pointer instrument scale of the to-be-tested image; calculating a matching degree of feature data of each key point on the to-be-tested image and feature data of a key point corresponding to the to-be-tested image on a template image, and obtaining an arc-shaped area containing a pointer based on the matching degree to identify a pointer reading in the arc-shaped area; obtaining an arc-shaped area containing a pointer based on the matching degree to identify a pointer reading in the arc-shaped area, comprising: for each key point on the to-be-tested image, obtaining a plurality of matching degrees of feature data of the key point and feature data of a key point corresponding to the template image, and determining target feature data based on a maximum value in the plurality of matching degrees; the plurality of matching degrees correspond to matching degrees of the key point in different groups; inputting the target feature data corresponding to each key point on the to-be-tested image and the feature data of the key point corresponding to the template image into an attention map neural network to obtain a first feature matrix corresponding to the to-be-tested image and a second feature matrix corresponding to the template image; calculating a matching score of the first feature matrix and the second feature matrix, and sequentially searching for matching feature data based on the matching score; mapping the matching feature data to the to-be-tested image to obtain an arc-shaped area containing a pointer, and positioning a pointer position in the arc-shaped area by using an image vertical projection method to identify a corresponding pointer reading; mapping the matching feature data to the to-be-tested image to obtain an arc-shaped area containing a pointer, comprising: calculating matching pair information of the matching feature data; the matching pair information is position coordinates and a matching score of a key point on the to-be-tested image that matches a key point on the template image well; calculating a homography matrix based on the matching pair information, the homography matrix being used for projection to a corresponding template image; performing affine transformation on the homography matrix and coordinates of the key point on the template image to obtain an arc-shaped area containing a pointer.

2. The method of claim 1, wherein, The feature data of the key point comprises feature point data and descriptor data; inputting the to-be-tested image into a deep learning network model to obtain feature data of a plurality of key points, comprising: inputting the to-be-tested image and a template image corresponding to the to-be-tested image into a deep learning network model for dimension reduction processing to obtain first data corresponding to the to-be-tested image and second data corresponding to the template image; The first data and the second data are processed by using a loss function to obtain a plurality of feature point data and a plurality of descriptor data corresponding to the to-be-tested image and the template image respectively; the feature point data includes position coordinates of key points; and the descriptor data includes directions, corresponding radii and corresponding pixel points of the key points.

3. The method of claim 1, wherein, The matching degrees of the feature data of each key point on the to-be-tested image and the feature data of the key points on the template image corresponding to the to-be-tested image are calculated, including: The feature data of each key point on the to-be-tested image is input into a denoising autoencoder to screen out noise data in the feature data; The matching degrees of the feature data screened out of noise and the feature data of the key points on the template image corresponding to the to-be-tested image are calculated respectively.

4. The method of claim 1, wherein, The matching scores of the first feature matrix and the second feature matrix are calculated, and matching feature data is sequentially searched based on the matching scores, including: The inner product of the first feature matrix and the second feature matrix is calculated to obtain N sets of matching score sets, and each set of matching score sets includes M matching scores; the first feature matrix is N rows, and the second feature matrix is M columns; The maximum value corresponding feature data in the N sets of matching score sets is sequentially searched, and the N feature data found is filtered based on a filtering channel, to filter out data corresponding to key points that do not match the template image; The N feature data filtered is normalized to obtain matching feature data.

5. The method of claim 1, wherein, The pointer position in the arc-shaped region is located by using an image vertical projection method, and the corresponding pointer reading is identified, including: The arc-shaped region is preprocessed, and the preprocessed arc-shaped region is converted into a rectangular region by polar coordinate conversion; the preprocessing is used to remove interference data; The rectangular region is binarized, and the position coordinates of the projection pointer corresponding to the binarized region are obtained by a vertical projection method; The pointer position is obtained based on the position coordinates, and the pointer reading corresponding to the pointer position is identified based on a range range; the range range is used to identify the pointer reading corresponding to different regions, and each range range corresponds to a pointer reading.

6. A pointer instrument reading device, characterized in that The device includes: An acquisition module is configured to acquire a to-be-tested image, input the to-be-tested image into a deep learning network model, and obtain feature data of a plurality of key points; the plurality of key points are divided into a plurality of groups, each group of key points corresponds to an image under a certain rotation angle; each group of key points includes three first key points and three second key points, the three first key points construct a first circular arc, the three second key points construct a second circular arc, the radii of the first circular arc and the second circular arc are different but the centers are the same, and the two circular arcs constitute an arc-shaped region containing a pointer instrument scale of the to-be-tested image; A processing module is configured to calculate the matching degrees of the feature data of each key point on the to-be-tested image and the feature data of the key points on the template image corresponding to the to-be-tested image, and obtain an arc-shaped region containing a pointer based on the matching degrees, and identify a pointer reading in the arc-shaped region. The processing module is specifically configured to, for each key point on the to-be-tested image, acquire a plurality of matching degrees of feature data of the key point and feature data corresponding to key points on the template image, and determine target feature data based on a maximum value in the plurality of matching degrees; the plurality of matching degrees correspond to matching degrees of the key point in different groups; input the target feature data corresponding to each key point on the to-be-tested image and the feature data corresponding to key points on the template image into an attention map neural network to obtain a first feature matrix corresponding to the to-be-tested image and a second feature matrix corresponding to the template image; calculate a matching score of the first feature matrix and the second feature matrix, and sequentially find matching feature data based on the matching score; map the matching feature data to the to-be-tested image to obtain an arc-shaped region containing a pointer, and locate a pointer position in the arc-shaped region by using an image vertical projection method to identify a corresponding pointer reading; The processing module is specifically configured to calculate matching pair information of the matching feature data; the matching pair information is position coordinates and a matching score of a key point on the to-be-tested image that is well matched with a key point on the template image; calculate a homography matrix based on the matching pair information, the homography matrix being used for projection to a corresponding template image; perform affine transformation on the homography matrix and coordinates corresponding to key points on the template image to obtain an arc-shaped region containing a pointer.

7. An electronic device, comprising: Comprise: A processor, a memory and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor, and the computer program comprises instructions for executing the pointer instrument reading identification method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the pointer instrument reading identification method according to any one of claims 1-5.

Images