An industrial-grade image recognition and text matching method and system

Abstract

The application provides an industrial-grade image recognition and text matching method and a method thereof, and the application obtains an enhanced identification plate image through preprocessing of an obtained industrial equipment identification plate image to be queried; and performs text detection and recognition by using OCR; removes the duplicate of a cropped identification subject region image by using a double-layer hash value; and extracts visual texture features and semantic features by using a double-branch model; obtains a candidate result through coarse screening, checks the candidate result by using a double-layer hash value, calculates a text matching score of a query text of the query image and a candidate text of the checking result, and judges a matching result based on a final matching score and the text matching score. The application realizes super-high-precision and super-low-delay recognition and matching of industrial identification through lightweight model depth optimization, pure CPU inference heterogeneous acceleration and multi-scene adaptive matching, and the text recognition accuracy is not less than 99.2%, and the overall matching accuracy is not less than 96.5%.

Description

Technical Field

[0001] This invention relates to the field of machine vision and image recognition technology, and in particular to an industrial-grade image recognition and text matching method and system. Background Technology

[0002] In industrial operation and maintenance scenarios such as key industrial bases and data centers, maintenance personnel need to quickly and accurately verify the consistency between on-site signage and the backend standard database using mobile devices. This is a crucial step in ensuring the normal operation of industrial production equipment and the adherence to operational procedures. However, existing industrial signage recognition and matching solutions have many technical limitations and fail to meet the actual needs of industrial sites. 1) Industrial sites commonly suffer from problems such as backlighting, low light, equipment reflection, and direct strong light. Existing solutions lack targeted image correction methods, resulting in a significant decrease in OCR recognition accuracy. 2) Existing solutions only filter edge noise based on spatial location, without considering noise unique to industrial scenarios such as oil stains, equipment engravings, and dynamic watermarks. This results in a high rate of missed / false detections, interfering with subsequent matching. 3) Industrial signs are prone to wear, tilting, stretching and other deformations. The existing solution only performs simple coordinate sorting and does not correct the deformed text, resulting in disordered / misaligned multi-line text and distorted matching results. 4) The existing model is still too large, the inference speed is slow, and it has not been optimized for heterogeneous CPU architecture, resulting in poor compatibility with low-end devices.

[0003] Therefore, there is an urgent need to provide an industrial-grade image recognition and text matching method and system to solve the technical problems existing in the background technology. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an industrial-grade image recognition and text matching method and system to solve the technical problems existing in the background technology.

[0005] In a first aspect, the present invention provides an industrial-grade image recognition and text matching method, comprising the following steps: S1) Obtain the image of the industrial equipment sign to be queried, and then preprocess it to obtain the enhanced sign image; S2) Use OCR to perform text detection and recognition on the enhanced sign image to obtain the text content and the original text box list. Then, use spatial attention filtering to remove interfering text and obtain a set of valid text boxes. Use dynamic thresholding to reorder and correct the set of valid text boxes to obtain the corrected recognized text. S3) Calculate the minimum bounding box based on the effective text boxes, and expand the minimum bounding box according to the distribution density of the text boxes; crop the image using the expanded bounding box, retaining only the main label area; calculate the double hash value of the cropped main label area image, and remove duplicates from the cropped main label area image based on the double hash value; S4) The CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8 is used to perform CPU heterogeneous inference on the deduplicated identification subject area image, extracting visual texture features and semantic features respectively, fusing and outputting a composite feature vector and performing L2 normalization to obtain a standardized feature vector. S5) Perform a dot product operation between the standardized feature vector of the image to be queried and all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results; S6) Perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and remove candidate images with a Hamming distance greater than 3 to obtain the verification results; S7) Calculate the text matching score between the query text of the query image and the candidate text of the verification result. S8) Add the highest score in the coarse screening results to the text matching score to obtain the final matching score, and judge the matching result based on the final matching score and the text matching score.

[0006] Preferably, in step S1), the preprocessing includes: The image of the industrial equipment signage to be queried is scaled and divided into multiple sub-blocks, and the mean brightness and variance of brightness of each sub-block are calculated. Adaptive gamma correction is performed on sub-blocks with a mean brightness greater than 220 or a mean brightness less than 30. Then, the corrected sign image is subjected to global contrast-limited adaptive histogram equalization (CLAHE) to obtain the enhanced sign image.

[0007] Preferably, in step S2), text detection and recognition are performed on the enhanced sign image using OCR, as follows; S211) The enhanced sign image is detected using the text detection model DB.onnx, and a text region probability map is output. The text region probability map is then subjected to threshold segmentation, dilated connected component analysis and minimum bounding polygon fitting, and low confidence and abnormal size regions are filtered out. Finally, text candidate boxes for all text regions in the sign image are obtained. S212) The text candidate box is classified by direction using the cls.onnx model, and the classification result of the text candidate box is output; if the text candidate box region is determined to be in the 180° inverted direction, its four-point coordinates are rotated 180° to correct the coordinates and the coordinate order of the upper left, upper right, lower right and lower left is rearranged. S213) CRNN.onnx is used to perform character sequence inference on the orientation-corrected text candidate boxes. The ONNXRuntime inference engine performs pure CPU forward computation to recognize the text characters, outputs the character sequence results and recognition confidence, and maps the detection coordinates to the recognized text content to generate the original text box list.

[0008] Preferably, in step S2), spatial attention filtering is used to remove interfering text, as follows: S221) For each original text box, calculate the centroid coordinates of the original text box based on its geometric center. The original text box texture features are extracted using the gray-level co-occurrence matrix (GLCM), and the area ratio of the original text box is calculated. S222) Calculate the edge region threshold based on the enhanced sign image. , ;in, , The span and height of the enhanced sign image; The cosine similarity between the texture feature vector of the original text box and the texture feature vectors of all noise samples in the industrial noise feature library is calculated to obtain the highest texture matching degree between the original text box and the noise feature library. ; S223) Filter the original text boxes based on spatial location, texture matching, and area ratio, and delete interfering text boxes; If the centroid coordinates of the original text box Within the threshold range of the edge region, and with the highest texture matching degree And the text box area ratio If the value is ≥0.5%, it is considered a valid text box; S224) The valid text boxes are initially grouped according to their spatial position in the image to form a set of valid text boxes.

[0009] Preferably, in step S2), the reordering and correction of the set of valid text boxes using dynamic thresholds is as follows: S231) Sort the valid text boxes in the set of valid text boxes in ascending order according to the vertical centroid coordinates to obtain a preliminary vertically sorted text box sequence. (S232) Based on the text box sequence after initial vertical sorting, adjust the dynamic threshold by text line density. Traverse the initially vertically sorted sequence of text boxes. Starting from the second text box, compare the current text box with the previous text box in turn. If the vertical centroid difference between the two text boxes is less than the dynamic threshold, then the current text box is determined to belong to the previous text box, and the current text box is assigned to the line group of the previous text box; otherwise, the two text boxes are determined to belong to different lines, a new line group is created, and the current text box is assigned to the new line group. After traversal, several text line groups are obtained. S233) For the text boxes in each line group, the text baseline is fitted by the least squares method, and the coordinates of the deformed text boxes are corrected according to the baseline tilt angle to form the corrected line group; S234) Perform in-line sorting on each corrected line group and concatenate the text content of all lines to obtain the corrected recognition text.

[0010] Preferably, in step S3), a two-layer hash value is formed by sequentially concatenating the perceptual hash pHash and the differential hash dHas. .

[0011] Preferably, in step S4), the similarity between the two images is determined by Hamming distance, and non-originating images are excluded.

[0012] Preferably, in step S5), the convolutional layers of the CLIP ViT-B / 32 ONNX dual-branch model are quantized with INT4; the fully connected layers are quantized with INT8; and visual texture features are extracted through the first branch. Semantic features are extracted through the second branch, and visual texture features and semantic features are fused together using a weighted splicing fusion method.

[0013] Secondly, the present invention provides an industrial-grade image recognition and text matching system, comprising: The preprocessing module is used to obtain the enhanced sign image after preprocessing the image of the industrial equipment sign to be queried. The text detection and recognition module uses OCR to detect and recognize text in the enhanced sign image, obtaining the text content and the original text box list. Spatial attention filtering is used to remove interfering text, resulting in a set of valid text boxes. The set of valid text boxes is then reordered and corrected using dynamic thresholding to obtain the corrected recognized text. The image deduplication module calculates the minimum bounding box based on the effective text boxes, expands the minimum bounding box according to the distribution density of the text boxes, crops the image according to the expanded bounding box, and retains only the main label area; calculates the double hash value of the cropped main label area image, and performs deduplication on the cropped main label area image according to the double hash value; The feature extraction module is used to perform CLIP model standard normalization on the deduplicated image of the main identification region; and to perform pure CPU heterogeneous inference on the CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8, extracting visual texture features and semantic features respectively, fusing and outputting a composite feature vector and performing L2 normalization. The coarse screening module is used to perform a dot product operation between the standardized feature vector of the image to be queried and all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results. The verification module is used to perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and remove candidate images with a Hamming distance greater than 3 to obtain the verification result. The text matching score calculation module is used to calculate the text matching score between the query text of the query image and the candidate text of the verification result. The matching result determination module is used to add the highest score in the coarse screening results to the text matching score to obtain the final matching score, and to determine the matching result based on the final matching score and the text matching score.

[0014] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the industrial-grade image recognition and text matching method.

[0015] The beneficial effects of this invention are as follows: 1. This invention reduces the size of the CLIP dual-branch model to less than 45MB through INT4 and INT8 quantization, achieves pure CPU heterogeneous parallel inference, improves inference speed by more than 3 times, and the total time for single image recognition and matching does not exceed 1.0s, further improving the efficiency of visual texture feature and semantic feature extraction, while improving the integrity of feature representation through visual details and semantic information. 2. This invention performs coarse screening of feature vectors using dot product, selecting the top 3 candidate results in descending order of score. While ensuring screening accuracy, it significantly shortens the matching time and improves screening efficiency by more than 60%. The coarsely screened candidate results undergo double-layer hash verification to eliminate non-homologous candidates with a Hamming distance exceeding 3, further reducing the probability of false positives. After fine screening, the homology rate of the candidates reaches 100%. 3. This invention improves text matching accuracy by integrating three indicators: edit distance similarity, Jaccard character overlap, and semantic cosine similarity, taking into account the character structure, character overlap, and semantic consistency of the text. At the same time, it significantly improves matching accuracy and robustness by matching text matching scores and the final matching score. 4. This invention achieves high-precision deduplication of industrial identification images through Hamming distance, improving the accuracy of excluding non-originating images; 5. This invention calculates text line density and adaptively adjusts a dynamic threshold of 40%-60%. Combined with a threshold comparison between the vertical centroid difference of the text boxes and the minimum height, it achieves precise grouping of text lines, completely resolving the misjudgment problems of line merging and splitting caused by traditional fixed thresholds. The accuracy of line grouping is improved by over 40%. For text boxes in the same line, the least squares method is used to fit the text baseline. Based on the baseline tilt angle, a rotation transformation is performed on the coordinates of the deformed text boxes, effectively eliminating coordinate deviations caused by text tilting and stretching. Simultaneously, the text box deformation coefficient is retained for subsequent error correction, improving text coordinate correction accuracy to over 98%. Within each line, text boxes are sorted in ascending order according to their horizontal centroid, ensuring that the text splicing order is completely consistent with the actual text order of industrial logos, completely resolving the disorder problem caused by text deformation. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the method of an embodiment of the present invention; Figure 2 This is a schematic diagram of the text detection and recognition process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the process for filtering out interfering text in an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1 like Figure 1 As shown in the figure, this embodiment of an industrial-grade image recognition and text matching method includes the following steps: S1) Obtain the image of the industrial equipment sign to be queried, and preprocess it to obtain the enhanced sign image; In this embodiment, the preprocessing includes: The industrial equipment identification sign image to be queried is divided into multiple sub-blocks after being scaled to a resolution of 1920×1080 or lower, and the mean brightness and variance of the brightness of each sub-block are calculated; as follows: The image of the industrial equipment sign to be queried is converted into a grayscale image and uniformly divided into M×N=8×8 fixed-size sub-blocks according to the spatial domain; Each of the aforementioned average brightness values Represented as: ; In the formula, K This represents the total number of pixels within the sub-block. for A small block, coordinates grayscale value; The aforementioned luminance variance is expressed as: ; In the formula, The brightness variance is used to determine the uniformity of illumination in sub-blocks. The smaller the brightness variance, the more extreme the illumination and the greater the correction intensity.

[0018] For average brightness Greater than 220, or average brightness Sub-blocks smaller than 30 undergo adaptive gamma correction; that is: For average brightness Dark sub-blocks with a gamma value less than 30 Set as: For average brightness For overly bright sub-blocks with a gamma value greater than 220, Set as: The adaptive gamma correction is expressed as follows: ; In the formula, The corrected grayscale value; Then, the corrected sign image is subjected to global contrast-limited adaptive histogram equalization (CLAHE) processing, limiting the contrast gain to 2.0, to solve the image blurring problem caused by backlighting, low light, and reflections in industrial scenes; the details are as follows: The corrected sign image is divided into non-overlapping local windows, and histogram statistics and equalization are performed independently on each window. For each window Statistical grayscale distribution histogram: In the formula, The grayscale level in the current window is The number of pixels; Grayscale; This is the corrected image of the sign. As an indicator function, when coordinates The corrected grayscale value is equal to the target grayscale level. The value is 1 if the condition is met, and 0 otherwise. Calculate the maximum allowed number of pixels If the grayscale level in the current window Number of pixels The excess pixels are evenly distributed to other gray levels; otherwise, they remain unchanged, resulting in the clipped histogram. ; Based on the cropped histogram Calculate the cumulative distribution function (CDF), and then obtain the equalized grayscale value through mapping, i.e.: ; ; In the formula, Represents the grayscale histogram after cropping. From gray level 0 to gray level The cumulative distribution function value; Representing coordinates The intermediate grayscale value of the pixel after equalization; Total pixels of the window; express The cumulative distribution function value; This represents the minimum value of the cumulative distribution function of the histogram across all gray levels; By fusing the outputs of adjacent windows using bilinear interpolation, a smooth, block-free final enhanced image is obtained, namely: In the formula, coordinates The final grayscale value of the pixel; Indicates bilinear interpolation; for The four nearest local windows.

[0019] S2) Using the ONNX version of RapidOCR, text detection and recognition are performed on the enhanced sign image to obtain the text content and the original text box list. Spatial attention filtering is used to remove interfering text to obtain a set of valid text boxes. The set of valid text boxes is then reordered and corrected using dynamic thresholding to obtain the corrected recognized text. In this embodiment, the ONNX version of RapidOCR is used to perform text detection and recognition on the enhanced sign image, as detailed below; S211) Using the text detection model DB.onnx to process the enhanced sign image The system performs detection and outputs a text region probability map. The text region probability map is then subjected to threshold segmentation, dilated connected component analysis, and minimum bounding polygon fitting. Low-confidence and abnormally sized regions are filtered out to obtain the final sign image. Text candidate boxes for all text regions in the text; S212) The text candidate boxes are classified by orientation using the cls.onnx model, and the classification results of the text candidate boxes are output, namely 0° positive direction or 180° inverted direction; If the text candidate box area is determined to be in the positive direction of 0°, its original four-point coordinates are directly retained without correction; if the text candidate box area is determined to be in the inverted direction of 180°, its four-point coordinates are rotated and corrected by 180°, and the coordinate order of the upper left, upper right, lower right, and lower left is rearranged. S213) CRNN.onnx is used to perform character sequence inference on the orientation-corrected text candidate boxes. The ONNXRuntime inference engine performs pure CPU forward computation to recognize the text characters in sequence, and outputs the character sequence results and recognition confidence. The character sequence results are the text content corresponding to the corresponding text candidate boxes, and the detection coordinates are matched one by one with the recognized text content to generate the original text box list.

[0020] In this embodiment, spatial attention filtering is used to remove interfering text, resulting in a set of valid text boxes, as detailed below: S221) For each original text box, calculate the centroid coordinates of the original text box based on its geometric center. The original text box texture features are extracted using the Gray-Level Co-occurrence Matrix (GLCM), and the area ratio of the original text box is calculated. ; ; In the formula, ( 、( 、( 、( These are the coordinates of the top left, top right, bottom right, and bottom left of the text box, respectively. The image region corresponding to each original text box is cropped, and the texture feature vector of the image region corresponding to each original text box is extracted by the gray-level co-occurrence matrix (GLCM). The area ratio of the text box mentioned above This is the ratio of the pixel area of ​​the original text box to the pixel area of ​​the entire enhanced sign image; S222) Calculate the edge region threshold based on the enhanced sign image. , ;in, , The span and height of the enhanced sign image; The cosine similarity between the texture feature vector of the original text box and the texture feature vectors of all noise samples in the industrial noise feature library is calculated to obtain the highest texture matching degree between the original text box and the noise feature library. ; In this embodiment, the noise feature library is constructed as follows: Collect typical interference noise samples of signage images in scenarios such as industrial bases, computer rooms, and power stations, covering industrial noise such as watermarks, timestamps, equipment reflections, oil stains, lettering interference, and cable shadows. The sample size is no less than 5,000 images, covering noise patterns under different lighting, angles, and equipment types. The contrast, correlation, energy, entropy and other texture features of the noise region are extracted by using the gray-level co-occurrence matrix (GLCM) to form a 16-dimensional texture feature vector, which represents the visual texture attributes of the noise. The typical spatial distribution coordinate range of statistical noise in the image is analyzed, and the spatial pattern that industrial noise is mostly located at the image edge and four corners is summarized to form a coordinate distribution feature set. Calculate the area ratio of the noise region to the entire image, and statistically analyze the area pattern of industrial noise being mostly small and sporadic, to form an area ratio feature threshold. The texture feature vectors, coordinate feature sets, and area feature thresholds of all noise samples are normalized and mapped to a unified numerical range to construct a structured noise feature library.

[0021] S223) Filter the original text boxes based on spatial location, texture matching, and area ratio, and delete interfering text boxes; If the centroid coordinates of the original text box satisfy: If the text box is identified as being interfered with by the image edge, it will be discarded. If the highest texture matching degree If the original text box is highly similar to the industrial noise texture, it is considered a distracting text box and is discarded. If the area of ​​the text box accounts for If the value is less than 0.5%, the original text box is determined to be a small, sporadic interference area with no valid text information, and is therefore considered an interference text box, which is then discarded. S224) The remaining valid text boxes are initially grouped according to their spatial position in the image to form a set of valid text boxes; In this embodiment, the corrected recognition text is obtained by reordering and correcting the set of valid text boxes using dynamic thresholds, as detailed below: S231) Sort the valid text boxes in the set of valid text boxes in ascending order according to the vertical centroid coordinates to obtain a preliminary vertically sorted text box sequence. S232) Based on the text box sequence after preliminary vertical sorting, adjust the dynamic threshold by text line density. ,Right now: When global text line density If the value is ≥5, then the dynamic adjustment coefficient is applied. ; When global text line density If the value is less than 5, then the dynamic adjustment coefficient is applied. ; Traverse the initially vertically sorted sequence of text boxes, starting from the second text box. Start, set the current text box sequentially with the previous text box To make a comparison, first calculate the minimum height of the two text boxes. Then calculate the dynamic threshold corresponding to the minimum value, and simultaneously calculate the vertical centroid difference between the two text boxes. ; If the vertical center of gravity is different Then determine the current text box With the previous text box Belongs to the previous text box Then the current text box Return to the previous text box The row group it belongs to; If the vertical center of gravity is different To determine if two text boxes belong to different lines, create a new line group and group the current text boxes. Reassigned to the new group; After the traversal is complete, several text line groups are obtained. ; S233), for each row group The text box within is fitted with a text baseline using the least squares method, and the baseline tilt angle is determined accordingly. Correct the coordinates of the deformed text box; In this embodiment, the text baseline refers to the virtual straight line containing the bottoms of all characters in the text line, used to characterize the tilt of the text line. The tilt angle of the baseline... This refers to the slant angle of the text line. The value range is -15° to 15°.

[0022] For each row group Extract the centroid coordinates of all text boxes within the group, and use the horizontal coordinates of the centroid coordinates as the independent variable and the vertical coordinates as the dependent variable. Fit the text baseline using the least squares method to obtain the following straight line equation: ; in, The slope of the line. The intercept; During the fitting process, the optimal solutions for the slope k and intercept b are obtained by minimizing the sum of squared errors using the least squares method; then, the optimal solutions for the slope k and intercept b are obtained by minimizing the sum of squared errors using the least squares method. Calculate the baseline tilt angle , Based on baseline tilt angle Perform a rotation transformation on the four coordinates of all text boxes within the row group to correct coordinate deviations caused by deformation, as follows: The average of the centroid coordinates of all text boxes within the row group is used as the rotation center; Perform a counter-clockwise rotation on the four coordinates of each text box. The rotation transformation is used to offset the baseline tilt angle, ensuring that the text baseline is parallel to the horizontal direction after rotation; Calculate the coordinate deviation of each text box before and after correction, i.e., the maximum horizontal deviation. Maximum deviation in the vertical direction And calculate the deformation coefficient. ,Right now: In the formula, This is the width of the text box. This is the height of the text box; After correcting the deformation of all text boxes within the corresponding row group, update the four coordinates of the text boxes to the corrected values. The deformation coefficient γ is retained to form the corrected row group. ; (S234) Perform in-line sorting on each corrected line group and concatenate the text content of all lines to obtain ordered and corrected recognition text, as follows: For each corrected row group Extract the horizontal centroid coordinates of all text boxes within the group. According to the horizontal centroid coordinates Sort in ascending order to obtain an inline ordered sequence of text boxes; Concatenate the text content arranged horizontally within each line group in a top-to-bottom order, preserving the natural spacing between the text during the concatenation process; Organize the spliced ​​text content and bind the corrected four-point coordinates and deformation coefficient of each text box. The confidence level is determined to form an ordered and corrected recognition text.

[0023] S3) Calculate the minimum bounding box based on the effective text boxes, and dynamically expand the minimum bounding box outward by 30%-60% according to the distribution density of the text boxes; crop the image according to the expanded bounding box, and retain only the main label area; calculate the double hash value of the cropped main label area image, and remove duplicates from the cropped main label area image according to the double hash value; In this embodiment, a two-layer hash value is formed by sequentially concatenating the perceptual hash pHash and the differential hash dHas. .

[0024] In this embodiment, the perceptual hash pHash is calculated as follows: After scaling the cropped image of the main identification area to a uniform size, the average grayscale value of all pixels is calculated. By comparing each pixel with the average grayscale value, if the pixel grayscale value is greater than or equal to the average grayscale value, it is recorded as 1; otherwise, it is recorded as 0. This process generates a binary sequence that represents the overall brightness distribution of the image, which is called the perceptual hash pHash.

[0025] The calculation of the differential hash dHas is as follows: traverse each pixel, compare the gray value of the current pixel with the gray value of the right neighboring pixel, if the gray value of the current pixel is greater than or equal to the gray value of the right neighboring pixel, then record it as 1, otherwise record it as 0, thereby generating a binary sequence that represents the local texture changes of the image, which is the differential hash dHas, used to determine the homology of image textures.

[0026] The similarity between two images is determined by Hamming distance, and images that are not from the same source are excluded. ; In the formula, , These are the two-layer hash values ​​of the two images to be compared; This is an XOR operation; like If the value is ≤3, the two images are determined to be from the same source, belonging to different shooting scenes of the same industrial logo. One of the valid images is retained to achieve deduplication. like The two images are determined to be non-originating images, belonging to different industrial identifiers, and are both retained without performing deduplication.

[0027] S4) Using the CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8, pure CPU heterogeneous inference is performed on the deduplicated identification subject region image to extract visual texture features and semantic features respectively, and the composite feature vector is fused and output and L2 normalization is performed. In this embodiment, the image of the main body region after deduplication is first scaled up, and then the pixel values ​​of the scaled image of the main body region are normalized to the [0,1] interval by the CLIP model. Then, a fixed mean is subtracted from each channel and divided by a fixed standard deviation to obtain the input tensor. In this embodiment, the convolutional layers of the CLIP ViT-B / 32 ONNX dual-branch model are quantized using INT4; the fully connected layers are quantized using INT8; and visual texture features are extracted through the first branch. Semantic features are extracted through the second branch. ; In this embodiment, a weighted splicing and fusion method is used to fuse visual texture features. With semantic features ,Right now: ; In the formula, , These are texture weights and semantic weights, respectively. It is a composite feature vector; The fused composite feature vector is normalized using the L2 norm to obtain the normalized standardized feature vector. ,Right now: In the formula, express The L2 norm; It is a local constant; Composite feature vector Total dimensions; Composite feature vector The first in One characteristic.

[0028] S5) The L2-normalized feature vector of the image to be queried. Perform a dot product operation with all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results; that is: ; In the formula, For coarse screening scoring; These are standard feature vectors in the database; S6) Perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and if the Hamming distance between the query image and any candidate result exceeds 3, directly remove the corresponding candidate image; obtain the verification result, i.e.: ; In the formula, To query the Hamming distance between the image and the candidate results; To query the double hash value of an image; This is the double hash value of the candidate results.

[0029] S7) Calculate the text matching score between the query text of the query image and the candidate text of the verification result. ,Right now: ; In the formula, This is the edit distance similarity score, with a value range of [0,1]. Indicates the score for the overlap of character sets; Score the semantic similarity of the texts; , , The weights for the corresponding scores; This represents the Lewinstein edit distance between the query text and the candidate texts; These represent the recognized text corresponding to the query image and the standard text corresponding to the candidate images in the database, respectively. This indicates that the longest one is selected; These are the character sets for the query text and the candidate text, respectively. These are the feature vectors of the query text and the candidate text, respectively. , respectively, are the L2 norms of the feature vectors of the query text and the candidate text.

[0030] S8) Match the highest score in the coarse screening results with the text matching score. Add them together to get the final matching score And based on the final match score Text matching score Determine the matching result; that is: ; In the formula, , These are the corresponding scoring weights; If the text matches the score <0.98, indicating a text mismatch; If the final matching score <0.92, considered low similarity; If the final matching score ≥0.92, and text matching score If the value is ≥0.98, the match is considered successful, and the standard text in the database and the original text recognized by OCR are output.

[0031] Example 2 This embodiment provides an industrial-grade image recognition and text matching system, including: The preprocessing module is used to obtain the enhanced sign image after preprocessing the image of the industrial equipment sign to be queried. The text detection and recognition module uses OCR to detect and recognize text in the enhanced sign image, obtaining the text content and the original text box list. Spatial attention filtering is used to remove interfering text, resulting in a set of valid text boxes. The set of valid text boxes is then reordered and corrected using dynamic thresholding to obtain the corrected recognized text. The image deduplication module calculates the minimum bounding box based on the effective text boxes, expands the minimum bounding box according to the distribution density of the text boxes, crops the image according to the expanded bounding box, and retains only the main label area; calculates the double hash value of the cropped main label area image, and performs deduplication on the cropped main label area image according to the double hash value; The feature extraction module is used to perform CLIP model standard normalization on the deduplicated image of the main identification region; and to perform pure CPU heterogeneous inference on the CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8, extracting visual texture features and semantic features respectively, fusing and outputting a composite feature vector and performing L2 normalization. The coarse screening module is used to perform a dot product operation between the standardized feature vector of the image to be queried and all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results. The verification module is used to perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and remove candidate images with a Hamming distance greater than 3 to obtain the verification result. The text matching score calculation module is used to calculate the text matching score between the query text of the query image and the candidate text of the verification result. The matching result determination module is used to add the highest score in the coarse screening results to the text matching score to obtain the final matching score, and to determine the matching result based on the final matching score and the text matching score.

[0032] Example 3 This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the industrial-grade image recognition and text matching method described in Embodiment 1.

[0033] In this embodiment, the memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. A processor, coupled to the memory, is used to execute computer programs stored in the memory.

[0034] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form.

[0035] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. An industrial-grade image recognition and text matching method, characterized in that, The steps include the following: S1) Obtain the image of the industrial equipment sign to be queried, and then preprocess it to obtain the enhanced sign image; S2) Use OCR to perform text detection and recognition on the enhanced sign image, and use spatial attention to filter out interfering text to obtain a set of effective text boxes; use dynamic thresholding to reorder and correct the set of effective text boxes to obtain the corrected recognized text. S3) Cropping the image using an expanded bounding box, retaining only the main label area; calculating the double hash value of the cropped main label area image, and deduplicating the cropped main label area image based on the double hash value; S4) The CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8 is used to perform CPU heterogeneous inference on the deduplicated identification subject area image, extracting visual texture features and semantic features respectively, fusing and outputting a composite feature vector and performing L2 normalization to obtain a standardized feature vector. S5) Perform a dot product operation between the standardized feature vector of the image to be queried and all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results; S6) Perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and remove candidate images with a Hamming distance greater than 3 to obtain the verification results; S7) Calculate the text matching score between the query text of the query image and the candidate text of the verification result. S8) Add the highest score in the coarse screening results to the text matching score to obtain the final matching score, and judge the matching result based on the final matching score and the text matching score.

2. The industrial-grade image recognition and text matching method according to claim 1, characterized in that: In step S1), the preprocessing includes: The image of the industrial equipment signage to be queried is scaled and divided into multiple sub-blocks, and the mean brightness and variance of brightness of each sub-block are calculated. Adaptive gamma correction is performed on sub-blocks with a mean brightness greater than 220 or a mean brightness less than 30. Then, the corrected sign image is subjected to global contrast-limited adaptive histogram equalization (CLAHE) to obtain the enhanced sign image.

3. The industrial-grade image recognition and text matching method according to claim 2, characterized in that: In step S2), OCR is used to perform text detection and recognition on the enhanced sign image, as detailed below; S211) The enhanced sign image is detected using the text detection model DB.onnx, and a text region probability map is output. The text region probability map is then subjected to threshold segmentation, dilated connected component analysis and minimum bounding polygon fitting, and low confidence and abnormal size regions are filtered out to obtain text candidate boxes for all text regions in the sign image. S212) The text candidate box is classified by direction using the cls.onnx model, and the classification result of the text candidate box is output; if the text candidate box region is determined to be in the 180° inverted direction, its four-point coordinates are rotated 180° to correct the coordinates and the coordinate order of the upper left, upper right, lower right and lower left is rearranged. S213) CRNN.onnx is used to perform character sequence inference on the orientation-corrected text candidate boxes. The ONNXRuntime inference engine performs pure CPU forward computation to recognize the text characters, outputs the character sequence results and recognition confidence, and maps the detection coordinates to the recognized text content to generate the original text box list.

4. The industrial-grade image recognition and text matching method according to claim 3, characterized in that: In step S2), spatial attention filtering is used to remove interfering text, as detailed below: S221) For each original text box, calculate the centroid coordinates of the original text box based on the geometric center, extract the texture features of the original text box through the gray-level co-occurrence matrix (GLCM), and calculate the area ratio of the original text box. S222) Calculate the edge region threshold based on the enhanced sign image; The cosine similarity between the texture feature vector of the original text box and the texture feature vector of all noise samples in the industrial noise feature library is calculated to obtain the highest texture matching degree between the original text box and the noise feature library. S223) Filter the original text boxes based on spatial location, texture matching, and area ratio, and delete interfering text boxes; If the centroid coordinates of the original text box are within the threshold range of the edge region, and the highest texture matching degree is less than If the area of ​​a text box is ≥0.5%, it is considered a valid text box. S224) The valid text boxes are initially grouped according to their spatial position in the image to form a set of valid text boxes.

5. The industrial-grade image recognition and text matching method according to claim 4, characterized in that: In step S2), the reordering and correction of the set of valid text boxes using dynamic thresholds is as follows: S231) Sort the valid text boxes in the set of valid text boxes in ascending order according to the vertical centroid coordinates to obtain a preliminary vertically sorted text box sequence. (S232) Based on the text box sequence after initial vertical sorting, adjust the dynamic threshold by text line density. Traverse the initially vertically sorted sequence of text boxes. Starting from the second text box, compare the current text box with the previous text box in turn. If the vertical centroid difference between the two text boxes is less than the dynamic threshold, then the current text box is determined to belong to the previous text box, and the current text box is assigned to the line group of the previous text box; otherwise, the two text boxes are determined to belong to different lines, a new line group is created, and the current text box is assigned to the new line group. After traversal, several text line groups are obtained. S233) For the text boxes in each line group, the text baseline is fitted by the least squares method, and the coordinates of the deformed text boxes are corrected according to the baseline tilt angle to form the corrected line group; S234) Perform in-line sorting on each corrected line group and concatenate the text content of all lines to obtain the corrected recognition text.

6. The industrial-grade image recognition and text matching method according to claim 1, characterized in that: In step S3), a two-layer hash value is formed by sequentially concatenating the perceptual hash pHash and the differential hash dHas. .

7. The industrial-grade image recognition and text matching method according to claim 1, characterized in that: In step S4), the similarity between the two images is determined by Hamming distance, and non-originating images are excluded.

8. The industrial-grade image recognition and text matching method according to claim 1, characterized in that: In step S5), the convolutional layers of the CLIP ViT-B / 32 ONNX dual-branch model are quantized using INT4; the fully connected layers are quantized using INT8; and visual texture features are extracted through the first branch. Semantic features are extracted through the second branch, and visual texture features and semantic features are fused together using a weighted splicing fusion method.

9. An industrial-grade image recognition and text matching system, characterized in that, include: The preprocessing module is used to obtain the enhanced sign image after preprocessing the image of the industrial equipment sign to be queried. The text detection and recognition module uses OCR to detect and recognize text in the enhanced sign image, obtaining the text content and the original text box list. Spatial attention filtering is used to remove interfering text, resulting in a set of valid text boxes. The set of valid text boxes is then reordered and corrected using dynamic thresholding to obtain the corrected recognized text. The image deduplication module calculates the minimum bounding box based on the effective text boxes, expands the minimum bounding box according to the distribution density of the text boxes, crops the image according to the expanded bounding box, and retains only the main label area; calculates the double hash value of the cropped main label area image, and performs deduplication on the cropped main label area image according to the double hash value; The feature extraction module is used to perform CLIP model standard normalization on the deduplicated image of the main identification region; and to perform pure CPU heterogeneous inference on the CLIP ViT-B / 32 ONNX dual-branch model quantized by INT4 and INT8, extracting visual texture features and semantic features respectively, fusing and outputting a composite feature vector and performing L2 normalization. The coarse screening module is used to perform a dot product operation between the standardized feature vector of the image to be queried and all standard feature vectors in the standard database, and select the top 3 with the highest scores as candidate results. The verification module is used to perform double-layer hash value verification on the candidate results, calculate the Hamming distance between the query image and the candidate results, and remove candidate images with a Hamming distance greater than 3 to obtain the verification result. The text matching score calculation module is used to calculate the text matching score between the query text of the query image and the candidate text of the verification result. The matching result determination module is used to add the highest score in the coarse screening results to the text matching score to obtain the final matching score, and to determine the matching result based on the final matching score and the text matching score.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the industrial-grade image recognition and text matching method as described in any one of claims 1-8.

Images