A FAKAZE-POCR digital instrument detection and recognition method

Abstract

The present application relates to a FAKAZE‑POCR digital instrument detection and recognition method. The method comprises: extracting key features of the industrial digital instrument image to be identified and the template image data set according to a contrast threshold, matching the obtained two FAKAZE features with the key features of the image, and calculating the optimal single mapping transformation matrix between the two FAKAZE features by the robust algorithm of PROSAC; obtaining the reading display area of ​​the digital instrument in the industrial digital instrument image to be identified according to the optimal single mapping transformation matrix and extracting the image of the region of interest; integrating the text recognition detection architecture based on the hardware NPU neural network processing accelerator; performing DB detection on the image of the region of interest according to the DB detection module, and performing text recognition on the text detection area using the CRNN text recognition module. The use of this method can improve the accuracy and efficiency of digital instrument detection and recognition.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a FAKAZE-POCR digital instrument detection and recognition method. Background Art

[0002] With the continuous development of science and technology, the scale of industrial production continues to expand. In today's digital age, digital instruments play a vital role in the field of industrial production. They also have important applications in the aerospace field. They are widely used to monitor and control various production processes. However, traditional digital instrument readings usually need to rely on manual interpretation and recording, which not only has the risk of human error, but also limits production efficiency and data accuracy. In addition, in some special working environments, such as high temperature, high pressure, high radiation, etc., manual operation readings are not only inefficient, but also have great safety hazards.

[0003] In order to solve the above problems, in recent years, with the continuous development of computer vision, image processing, pattern recognition and other technologies, the automatic recognition technology of industrial digital instrument readings has gradually become a research hotspot. This technology uses computer vision technology to collect, process and recognize instrument images, and realizes automatic extraction and display of instrument readings, which can not only greatly improve the accuracy and efficiency of readings, but also reduce the risks and costs of manual operations.

[0004] At present, the research on automatic recognition technology for industrial digital instrument readings has made certain progress. Domestic and foreign scholars and research institutions have conducted a lot of research and practice in image acquisition, image preprocessing, feature extraction, classification and recognition, and proposed a series of effective algorithms and models. At the same time, with the continuous development of artificial intelligence technologies such as deep learning, the research on automatic recognition technology for industrial digital instrument readings has also shown new development trends. For example, image recognition algorithms based on deep learning can accurately identify and extract instrument readings under complex background conditions, thereby further improving the accuracy and efficiency of readings.

[0005] In summary, the research on automatic recognition technology of industrial digital instrument readings has important theoretical value and practical significance. Summary of the invention

[0006] Based on this, it is necessary to provide a FAKAZE-POCR digital instrument detection and identification method that can improve the accuracy and efficiency of digital instrument detection and identification in response to the above technical problems.

[0007] A FAKAZE-POCR digital instrument detection and identification method, the method comprising:

[0008] Acquire the industrial digital instrument image to be identified; construct a template image dataset of various digital instruments;

[0009] Extract key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold, perform image key feature matching on the two obtained FAKAZE features, and calculate the optimal single mapping transformation matrix between the two FAKAZE features through the robust algorithm of PROSAC;

[0010] According to the optimal single mapping transformation matrix, the reading display area of ​​the digital instrument in the industrial digital instrument image to be identified is obtained and the image of the area of ​​interest is extracted;

[0011] A text recognition detection architecture is integrated based on a hardware NPU neural network processing accelerator; the text recognition detection architecture includes a DB detection module and a CRNN text recognition module;

[0012] The DB detection module is used to perform DB detection on the image of the region of interest to obtain a text detection region; and the CRNN text recognition module is used to perform text recognition on the text detection region to obtain OCR characters.

[0013] The above-mentioned FAKAZE-POCR digital instrument detection and recognition method, the present application extracts key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold. By extracting key features through the contrast threshold, these important information can be accurately captured and irrelevant background interference can be effectively eliminated.

[0014] The FAKAZE features extracted based on the contrast threshold have local invariance. When the image is rotated, scaled, etc., it can stably describe key information, so that the feature extraction of the instrument at different angles and under different conditions can maintain accuracy, providing a reliable basis for subsequent key feature matching of the image. At the same time, the robust algorithm of PROSAC is used to calculate the optimal single mapping transformation matrix between the two FAKAZE features to ensure the accuracy and stability of feature matching. Based on the hardware NPU neural network processing accelerator integrated text recognition detection architecture, the DB detection module and the CRNN text recognition module work together to detect and recognize the image of the area of ​​interest, and finally obtain accurate OCR characters, which greatly improves the accuracy and efficiency of digital instrument detection and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is an application scenario diagram of a FAKAZE-POCR digital instrument detection and identification method in one embodiment;

[0016] Figure 2 The figure is a flow chart of a method for automatic character recognition in one embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] In one embodiment, Figure 1 As shown, a FAKAZE-POCR digital instrument detection and identification method is provided, comprising the following steps:

[0019] Step 102, obtain the industrial digital instrument image to be identified; construct a template image data set of multiple digital instruments; extract key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold, perform image key feature matching on the two obtained FAKAZE features, and calculate the optimal single mapping transformation matrix between the two FAKAZE features through the robust algorithm of PROSAC.

[0020] Building a template image dataset of various digital instruments can provide rich reference standards for subsequent feature matching. Due to the wide variety of digital instruments in industrial environments, by building such a dataset, instruments of different models and styles can be covered, so that the system can adapt to various possible instrument types, thereby improving the versatility and accuracy of overall recognition. Key feature extraction based on contrast thresholds can focus on areas with significant visual differences in the image. In industrial digital instrument images, key information such as numbers and scales usually have a significant difference in contrast with the background. In this way, these important features related to the readings can be accurately captured to avoid interference from irrelevant background information. For situations such as changes in instrument shooting angles and uneven lighting that may occur in industrial environments, FAKAZE features can effectively reduce the impact of these factors on feature extraction, ensuring that the extracted features can accurately represent the key information of the instrument.

[0021] After extracting the FAKAZE features of the industrial digital instrument image to be detected and identified, the FAKAZE features of the template image of the corresponding type of instrument in the template data set are extracted at the same time, and then a Brute-Force feature matcher is established. The process is to select a feature point from the extracted feature points of the template image, and then calculate the distance between the feature points with each feature point of the image to be detected and identified in turn, so as to obtain multiple pairs of key feature points that match each other. After obtaining the key feature point pairs that match each other, the optimal single mapping transformation matrix between the feature point pairs of the two images is calculated by the robust algorithm of PROSAC. Matching the FAKAZE features of the image to be identified and the template image data set can establish an association between the two. Through this matching, the corresponding relationship between the instrument in the image to be identified and the template instrument can be found, thereby providing a basis for the subsequent positioning of the reading area.

[0022] In actual industrial environments, the image acquisition process may be interfered by various factors, such as partial occlusion, image blur, etc., which may cause some errors or inaccuracies in feature matching. The PROSAC algorithm screens and optimizes matching points to find the most reliable single mapping transformation relationship, thereby improving the accuracy and stability of matching and laying a solid foundation for accurately obtaining the reading display area.

[0023] Step 104 , obtaining the reading display area of ​​the digital instrument in the industrial digital instrument image to be identified according to the optimal single mapping transformation matrix and extracting the image of the region of interest.

[0024] The reading display area of ​​the digital instrument in the industrial digital instrument image to be identified is obtained according to the optimal single mapping transformation matrix. According to the relationship obtained by the previous feature matching, the position of the reading display area is accurately found in the image to be identified, avoiding the errors that may be caused by manual or other rough positioning methods, and improving the accuracy and efficiency of positioning.

[0025] Step 106, integrating a text recognition detection architecture based on a hardware NPU neural network processing accelerator; the text recognition detection architecture includes a DB detection module and a CRNN text recognition module.

[0026] Using hardware NPU neural network processing accelerator can significantly improve the computing speed. The hardware NPU neural network processing accelerator of the embedded platform realizes the accelerated reasoning process of the pre-trained model of DB detection and CRNN text recognition. First, the pre-trained model (.nb format file) is converted into an RKNN model through the RKNN Toolkit, and then the reasoning process of the detection and recognition algorithm is executed through the RKNN API function interface to achieve software and hardware collaborative optimization, thereby improving the execution rate of the detection and recognition algorithm. In the text recognition process, a large number of neural network calculations are involved, such as the calculation of the DB detection module and the CRNN text recognition module. The acceleration function of the NPU can greatly shorten the processing time and improve the overall recognition efficiency.

[0027] Step 108, performing DB detection on the image of the region of interest according to the DB detection module to obtain a text detection region; performing text recognition on the text detection region using the CRNN text recognition module to obtain OCR characters.

[0028] According to the DB detection module, DB detection is performed on the image of the region of interest. The text area in the image can be effectively detected through a deep learning algorithm. The backbone network can be selected from the ShuffleNetV2 series, MobileNetV1 series, MobileNetV2 series and MobileNetV3 series. The Head network in DB text detection adopts a structure similar to FPN (Feature Pyramid Network). The main function of this structure is to fuse feature maps of multiple resolutions extracted by the backbone network, thereby improving the detection effect of text targets of different scales. A 1x1 convolution kernel was selected in the application to better integrate with the features. In addition, the number of feature channels of feature maps of different resolutions needs to be converted to the same number of feature channels, so that the predicted output of the network's probability map and threshold map will be affected by the number of feature channels. This application intends to use a feature channel number of 96 to take into account the size of the model, inference speed, and detection accuracy.

[0029] This application uses the CRNN text recognition module to identify the detected text area, which can effectively process sequence data, such as text character sequences, and accurately identify OCR characters for complex text situations such as numbers and letter combinations that may appear in industrial digital instrument readings, further improving the accuracy of text recognition.

[0030] In the above-mentioned FAKAZE-POCR digital instrument detection and recognition method, the present application extracts key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold. By extracting key features through the contrast threshold, these important information can be accurately captured and irrelevant background interference can be effectively eliminated. The FAKAZE features extracted according to the contrast threshold have local invariance. When the image undergoes changes such as rotation and scaling, they can stably describe key information, so that the feature extraction of the instrument at different angles and under different conditions can maintain accuracy, providing a reliable basis for subsequent image key feature matching. At the same time, the robust algorithm of PROSAC is used to calculate the optimal single mapping transformation matrix between the two FAKAZE features to ensure the accuracy and stability of feature matching. Based on the hardware NPU neural network processing accelerator integrated text recognition detection architecture, the DB detection module and the CRNN text recognition module work together to detect and identify the image of the area of ​​interest, and finally obtain accurate OCR characters, which greatly improves the accuracy and efficiency of digital instrument detection and recognition.

[0031] In one embodiment, key features are extracted from the industrial digital instrument image to be identified and the template image dataset according to a contrast threshold, including:

[0032] The local texture coefficient around each feature point of the industrial digital instrument image to be identified and the template image in the template image dataset is calculated. According to the local texture coefficient, the image is divided into different texture levels and the contrast threshold is dynamically adjusted. A contrast threshold is set for each texture level. When the local texture coefficient is less than 0.01, a contrast threshold of 0.01 is used to increase the number of feature points extracted. When the local texture coefficient is greater than or equal to 0.01, a contrast threshold of 0.03 is used to improve the quality of feature points. The value range of the contrast threshold is 0.01 to 0.05.

[0033] By calculating the local texture coefficient around each feature point of the template image, the local texture characteristics of the image can be accurately reflected. This allows different texture areas to be clearly divided in the instrument image. This provides an important basis for subsequent recognition. At the same time, different texture levels are divided according to the local texture coefficient, and a corresponding contrast threshold is set for each texture level. This dynamic adjustment method enables the image to extract features with optimal contrast in different texture areas. For example, for areas with smaller texture coefficients, a lower contrast threshold is used to increase the number of feature point extractions in the area. For areas with larger texture coefficients, a higher contrast threshold is used to improve the quality of the feature points. This directly improves the accuracy of instrument image feature extraction, thereby improving recognition accuracy.

[0034] Using a lower contrast threshold of 0.01 in areas where the local texture coefficient is less than 0.01 can increase the number of feature points extracted. This is because in areas with smoother textures or fewer details, a lower contrast threshold can enable more feature points to be detected. For example, in some background areas of instruments, the texture is relatively simple. By lowering the contrast threshold, more feature points can be extracted, thereby enriching the feature information of the image.

[0035] Using a higher contrast threshold of 0.03 in areas where the local texture coefficient is not less than 0.01 can improve the quality of feature points. This is because in areas with complex textures or rich details, a higher contrast threshold can make feature points more prominent, thereby enhancing the stability and reliability of feature points. For example, in the digital and scale areas of instruments, where the texture is complex, a higher contrast threshold can make feature points more accurately reflect the texture characteristics of these areas, improving the accuracy of recognition.

[0036] In one embodiment, the local texture coefficients of the industrial digital instrument image to be identified and the template image in the template image data set are calculated as:

[0037] ;

[0038] in, m、n are the coordinates of the key points in the image, N is equal to 7 pixels, is the grayscale value of the pixel in scale space, and yes N×N The average gray value in the template, r Indicates the image size. , Indicates that in ( m,n ) is the center and the side length is 2 r +1 is the coordinate vector traversed in the area. , Indicates that in ( , ) is the center N×N A vector of coordinates within the template's bounds.

[0039] In one embodiment, the optimal single mapping transformation matrix between two FAKAZE features is calculated by the robust algorithm of PROSAC, including:

[0040] The optimal single mapping transformation matrix between two FAKAZE features is calculated by PROSAC's robust algorithm:

[0041] ;

[0042] in, Indicates the coordinates of the feature points in the original image that match the corresponding feature points. Represents the coordinates of the corresponding feature points in the template image in the image, Indicates i The coefficients for the matching points, represents the optimal single mapping transformation matrix.

[0043] In one embodiment, DB detection is performed on the region of interest image according to the DB detection module to obtain the text detection region, including:

[0044] The backbone network is trained using the cosine learning rate adjustment strategy, and the FPGM model pruning technology is used to perform lightweight compression optimization on the backbone network. The filter of each convolutional layer is regarded as a point in the Euclidean space, and the center value of these points is calculated. The filters closer to the center are preferentially pruned to obtain a trained backbone network.

[0045] Extract feature maps of various resolutions from the image of the region of interest based on the trained backbone network, input the feature maps of various resolutions into the feature pyramid network constructed by 1x1 convolution kernel for feature fusion, convert the number of feature channels of feature maps of different resolutions into the same number of feature channels, and obtain the initial detection area;

[0046] Perform image enhancement on the initial detection area to obtain the text detection area.

[0047] In one embodiment, the CRNN text recognition module is used to perform text recognition on the text detection area to obtain OCR characters, including:

[0048] In the convolution layer, the input image is feature extracted to obtain a feature map. The feature map is passed through the recurrent layer and the RNN architecture is used to predict each frame in the feature sequence to obtain the predicted probability distribution. In the transcription layer, the bidirectional LSTM is used to convert these predicted probability distributions into a tag sequence to obtain OCR characters.

[0049] In a specific embodiment, using the CRNN text recognition module to perform text recognition on the text detection area includes: The first module uses a lightweight MobileNetv3 CNN network to extract the features of the input image. The height of the input image is set to 32, and the width can be any length. After CNN processing, the height of the feature map is scaled to 1. The second module Im2Seq is responsible for converting the feature map extracted by CNN into the feature vector sequence shape required by RNN. The third module uses a bidirectional LSTM to predict these feature sequences to learn each feature vector in the sequence and output its predicted label distribution. This process is equivalent to treating the width of the feature vector as the time dimension in LSTM. The fourth module obtains the final prediction result of the model through the fully connected layer. Finally, the fifth module CTC transcription layer is used to decode the prediction result output by the model to obtain the final output.

[0050] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0051] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0052] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A FAKAZE-POCR digital instrument detection and identification method, characterized in that: The method comprises: Acquire the industrial digital instrument image to be identified; construct a template image dataset of various digital instruments; Extract key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold, perform image key feature matching on the two obtained FAKAZE features, and calculate the optimal single mapping transformation matrix between the two FAKAZE features through the robust algorithm of PROSAC; According to the optimal single mapping transformation matrix, the reading display area of ​​the digital instrument in the industrial digital instrument image to be identified is obtained and the image of the area of ​​interest is extracted; A text recognition detection architecture is integrated based on a hardware NPU neural network processing accelerator; the text recognition detection architecture includes a DB detection module and a CRNN text recognition module; Perform DB detection on the image of the region of interest according to the DB detection module to obtain a text detection region; perform text recognition on the text detection region using the CRNN text recognition module to obtain OCR characters; The optimal single mapping transformation matrix between two FAKAZE features is calculated using PROSAC’s robust algorithm, including: The optimal single mapping transformation matrix between two FAKAZE features is calculated by PROSAC's robust algorithm: in, Indicates the coordinates of the feature points in the original image that match the corresponding feature points. Represents the coordinates of the corresponding feature points in the template image in the image, Indicates The coefficients for the matching points, represents the optimal single mapping transformation matrix; Performing DB detection on the region of interest image according to the DB detection module to obtain a text detection region includes: The backbone network is trained using the cosine learning rate adjustment strategy, and the FPGM model pruning technology is used to perform lightweight compression optimization on the backbone network. The filter of each convolutional layer is regarded as a point in the Euclidean space, and the center value of these points is calculated. The filters closer to the center are preferentially pruned to obtain a trained backbone network. Extract feature maps of various resolutions from the image of the region of interest based on the trained backbone network, input the feature maps of various resolutions into the feature pyramid network constructed by 1x1 convolution kernel for feature fusion, convert the number of feature channels of feature maps of different resolutions into the same number of feature channels, and obtain the initial detection area; Performing image enhancement on the initial detection area to obtain a text detection area; The CRNN text recognition module is used to perform text recognition on the text detection area to obtain OCR characters, including: The convolutional layer extracts features from the input image to obtain a feature map; the recurrent layer of the feature map uses an RNN architecture to predict each frame in the feature sequence to obtain a predicted probability distribution; the bidirectional LSTM is used in the transcription layer to convert these predicted probability distributions into a tag sequence to obtain OCR characters.

2. The method according to claim 1, characterized in that Extracting key features of the industrial digital instrument image to be identified and the template image data set according to the contrast threshold includes: The local texture coefficient around each feature point of the industrial digital instrument image to be identified and the template image in the template image data set is calculated, and the image is divided into different texture levels according to the local texture coefficient and the contrast threshold is dynamically adjusted. A contrast threshold is set for each texture level, and a contrast threshold of 0.01 is used where the local texture coefficient is less than 0.01 to increase the number of feature points to be extracted, and a contrast threshold of 0.03 is used where the local texture coefficient is greater than or equal to 0.01 to improve the quality of the feature points; the value range of the contrast threshold is 0.01 to 0.

05.

3. The method according to claim 2, characterized in that Calculate the local texture coefficient of the industrial digital instrument image to be identified and the template image in the template image data set as in, m、n are the coordinates of the key points in the image, N is equal to 7 pixels, is the grayscale value of the pixel in scale space, and yes N×N The average gray value in the template, r Indicates the image size. , Indicates that in ( m,n ) is the center and the side length is 2 r +1 is the coordinate vector traversed in the area. , Indicates that in ( ， ) is the center N×N A vector of coordinates within the template's bounds.

Images