Intelligent identification method and device for steel plate code spraying based on multi-camera cooperation

Abstract

The application provides a steel plate code spraying intelligent recognition method and device based on multi-camera cooperation, relates to the technical field of computer vision, and solves the technical problems that the completeness of characters is difficult to guarantee and the recognition robustness is insufficient under strong reflection, blur or shielding conditions in the prior art. The method comprises the following steps: acquiring a plurality of original images of a code spraying area on the same object through a plurality of image acquisition devices; performing image alignment on the original images based on camera calibration to obtain a plurality of aligned images; detecting and cutting the characters in each aligned image to obtain a plurality of character images; performing quality evaluation on the plurality of character images to obtain image quality scores; repairing and combining the plurality of character images according to the image quality scores to obtain a combined image; and performing character recognition on the combined image to obtain the object code spraying. The application is used in the steel plate code spraying recognition process.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and device for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration. Background Technology

[0002] Existing technologies mainly use a single camera to acquire images and attempt to improve the recognizability of character regions through image enhancement, local threshold segmentation, or traditional morphological processing. However, such methods are difficult to deal with the problem of loss of character structural information caused by strong reflections on the steel plate surface, local blurring, or occlusion. Although some solutions have introduced multi-camera systems, they only use simple image stitching or voting strategies for recognition and lack quality assessment and feature-level collaborative repair mechanisms for multi-view images, resulting in insufficient character integrity and a high misrecognition rate. Summary of the Invention

[0003] This application provides a method and device for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration, which solves the technical problems of insufficient character integrity and recognition robustness in the existing technology under strong reflective, blurry or occluded conditions.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, a method for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration is provided, including:

[0006] Several original images of a coding area on the same object are acquired using several image acquisition devices; wherein, the coding area is an area that includes several characters;

[0007] Based on camera calibration, the original images are aligned to obtain several aligned images;

[0008] The characters in each aligned image are detected and cropped to obtain several character images;

[0009] A quality assessment is performed on several character images to obtain an image quality score;

[0010] Based on the image quality score, several character images are repaired and combined to obtain a combined image;

[0011] Character recognition is performed on the combined image to obtain the object's inkjet code.

[0012] Based on the above technical solution, the intelligent recognition method for steel plate inkjet printing based on multi-camera collaboration provided in this application effectively overcomes the problem of missing character features caused by reflection, blurring, or occlusion in single-view imaging by using multi-view image collaborative acquisition and intelligent fusion: camera calibration and image alignment ensure the spatial consistency of multi-source data, and combined with character-level quality assessment and optimal feature extraction mechanism, the clearest and most complete area of ​​each character is accurately selected for dynamic repair and combination, which significantly improves the integrity and recognizability of the character area; finally, recognition is performed through high-quality combined images, which greatly enhances the anti-interference ability and recognition accuracy of the system in complex industrial environments and solves the problem of poor robustness of traditional methods.

[0013] In conjunction with the first aspect above, in one possible implementation, the quality assessment of several character images includes:

[0014] The character structure integrity of the character image is evaluated to obtain a character structure integrity score;

[0015] The reflectivity of characters in a character image is evaluated to obtain a character reflectivity score;

[0016] The blurriness of the characters in the image is evaluated to obtain a blurriness score.

[0017] The image quality score is calculated using an image quality formula; the expression for the image quality formula is:

[0018] ;

[0019] In the formula, For the normalized character structure integrity score, To normalize the character reflectivity score, The score represents the normalized character ambiguity level, with α, β, and γ being the weighting coefficients.

[0020] In conjunction with the first aspect above, in one possible implementation, the evaluation of the character structure integrity of the character image includes:

[0021] The skeleton of a character is extracted from a character image using a morphological thinning algorithm to obtain a character skeleton map;

[0022] Breakpoints are identified and counted in the character skeleton diagram to obtain the number of breakpoints;

[0023] The skeleton in the character skeleton diagram is decomposed into a point sequence by a tracking algorithm, and the approximate curvature of the point sequence is calculated to obtain a curvature sequence.

[0024] Calculate the standard deviation of the curvature sequence to obtain the degree of character deformation;

[0025] The number of breakpoints and the degree of character deformation are normalized and then weighted and summed to obtain the character structure integrity score.

[0026] In conjunction with the first aspect above, in one possible implementation, the evaluation of the reflectivity of the characters in the character image includes:

[0027] Binarize the character image to obtain a specular mask image;

[0028] Perform a logical AND operation between the pixels of the specular mask image and the character skeleton image, and count the number of overlapping pixels;

[0029] Count the number of pixels in the skeleton region of the character skeleton image to obtain the total number of pixels;

[0030] The ratio of overlapping pixels to the total number of pixels is used to obtain a score for the character's reflectivity.

[0031] In conjunction with the first aspect above, in one possible implementation, the binarization operation on the character image includes:

[0032] Set the window size N; where N is a positive integer;

[0033] The neighborhood weighted mean of each pixel in the character image is calculated based on the window size N, and the difference is taken with the preset constant C to obtain the initial threshold matrix;

[0034] By iterating through several character images and calculating the average of several thresholds corresponding to the same pixel coordinate position, a standard threshold matrix is ​​obtained.

[0035] The character image is converted into a binary image based on the standard threshold matrix and labeled as a specular mask image.

[0036] In conjunction with the first aspect above, in one possible implementation, the evaluation of the blurriness of the characters in the character image includes:

[0037] Perform a character masking operation on the character image to obtain the character mask region;

[0038] The sharpness response map is obtained by convolving the character image using a multi-directional kernel function.

[0039] Extract the response values ​​of the corresponding character mask regions from the sharpness response map to obtain the character response map;

[0040] Calculate the variance of the character response map to obtain the character ambiguity score.

[0041] In conjunction with the first aspect above, in one possible implementation, the convolution calculation of the character image using a multi-directional kernel function includes:

[0042] Several gradient operators in different directions are used to convolve with the character image to obtain several gradient response maps;

[0043] The maximum response value of each pixel in several gradient response maps is selected to form a maximum response map.

[0044] The sharpness response map is obtained by performing a second convolution on the maximum response map using a Laplacian kernel.

[0045] In conjunction with the first aspect above, in one possible implementation, the repair and combination of several character images based on image quality scores includes:

[0046] The highest image quality score is selected for each character to obtain the highest quality score;

[0047] The highest quality score is compared with a preset threshold.

[0048] When the highest quality score is greater than or equal to a preset threshold, the character image corresponding to the highest quality score is marked as the final image;

[0049] When the highest quality score is less than the preset threshold, the character image corresponding to the highest quality score is repaired to obtain the repaired image, and the repaired image is marked as the final image;

[0050] Several final images are combined according to their coordinate positions to obtain a composite image.

[0051] In conjunction with the first aspect above, in one possible implementation, the repair of the character image corresponding to the highest quality score includes:

[0052] Defect detection is performed on the character image to generate a mask of the defect region;

[0053] Calculate the image quality of the character image on the defect region mask to obtain the defect region quality score;

[0054] Calculate the image quality of the remaining character images in the defect area mask and select the maximum value to obtain the candidate image quality score;

[0055] The quality score of the defective region is compared with the quality score of the candidate image.

[0056] When the quality score of the defective region is greater than or equal to the quality score of the candidate image, the character image corresponding to the quality score of the defective region is marked as the repaired image.

[0057] When the quality score of the defective region is less than the quality score of the candidate image, the character image corresponding to the quality score of the candidate image is cropped according to the defective region mask to obtain the cropped region; the character image corresponding to the quality score of the defective region is replaced according to the cropped region to obtain the patch image, and the patch image is marked as the repaired image.

[0058] In a second aspect, an electronic device is provided, comprising: a communication unit and a processing unit; the communication unit is configured to acquire several original images of a coding area on the same object through several image acquisition devices; the processing unit is configured to perform image alignment on the original images based on camera calibration to obtain several aligned images; detect and crop characters in each aligned image to obtain several character images; perform quality assessment on the several character images to obtain image quality scores; repair and combine the several character images according to the image quality scores to obtain combined images; and perform character recognition on the combined images to obtain object coding.

[0059] Thirdly, this application provides an electronic device, including: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. This electronic device may be an electronic device or a chip within an electronic device.

[0060] Fourthly, this application provides a multi-camera collaborative intelligent recognition system for steel plate inkjet printing, comprising: an image acquisition device, a cloud computing device, and a network device; wherein, the image acquisition device is used to acquire several original images of the inkjet printing area on the same object through several image acquisition devices; the cloud computing device is used to perform image alignment on the original images based on camera calibration to obtain several aligned images; to detect and crop characters in each aligned image to obtain several character images; to perform quality evaluation on the several character images to obtain image quality scores; to repair and combine the several character images according to the image quality scores to obtain a combined image; to perform character recognition on the combined image to obtain the object inkjet printing; and the network device is used to transmit the data acquired by the image acquisition device to the cloud computing device.

[0061] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0062] In a sixth aspect, this application provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0063] This application provides a method and device for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration. By collaborative acquisition and intelligent fusion of multi-view images, it effectively overcomes the problem of missing character features caused by reflection, blurring, or occlusion in single-view imaging. It uses camera calibration and image alignment to ensure spatial consistency of multi-source data, and combines character-level quality assessment and optimal feature extraction mechanisms to accurately select the clearest and most complete areas of each character for dynamic repair and combination, significantly improving the integrity and recognizability of the character area. Finally, it uses high-quality combined images for recognition, greatly enhancing the system's anti-interference ability and recognition accuracy in complex industrial environments, and solving the problem of poor robustness of traditional methods.

[0064] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0065] Figure 1 A system architecture diagram of a steel plate inkjet printing intelligent recognition system based on multi-camera collaboration is provided for embodiments of this application;

[0066] Figure 2 A flowchart illustrating a method for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration, provided in an embodiment of this application;

[0067] Figure 3 A flowchart illustrating another intelligent steel plate marking recognition method based on multi-camera collaboration provided in this application embodiment;

[0068] Figure 4 A flowchart illustrating another intelligent steel plate marking recognition method based on multi-camera collaboration provided in this application embodiment;

[0069] Figure 5 A flowchart illustrating another intelligent steel plate marking recognition method based on multi-camera collaboration provided in this application embodiment;

[0070] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0071] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0073] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0074] The intelligent recognition method for steel plate inkjet printing based on multi-camera collaboration provided in this application embodiment can be applied to, for example... Figure 1 In the steel plate inkjet printing intelligent recognition system 100 based on multi-camera collaboration shown, such as Figure 1 As shown, the communication system includes: an image acquisition device 10, a cloud computing device 20, and a network device 30.

[0075] Among them, the image acquisition device 10 is used to acquire several original images of the coding area on the same object through several image acquisition devices.

[0076] The cloud computing device 20 is used to perform image alignment on the original image based on camera calibration to obtain several aligned images; to detect and crop characters in each aligned image to obtain several character images; to perform quality assessment on the several character images to obtain image quality scores; to repair and combine the several character images according to the image quality scores to obtain combined images; and to perform character recognition on the combined images to obtain object inkjet codes.

[0077] Network device 30 is used to transmit the data acquired by image acquisition device 10 to cloud computing device 20.

[0078] To address the technical problems of insufficient character integrity and recognition robustness under conditions of strong reflection, blurring, or occlusion in existing technologies, this application provides a method for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration. This method includes:

[0079] Several original images of a coding area on the same object are acquired using several image acquisition devices; wherein, the coding area is an area that includes several characters;

[0080] Based on camera calibration, the original images are aligned to obtain several aligned images;

[0081] The characters in each aligned image are detected and cropped to obtain several character images;

[0082] A quality assessment is performed on several character images to obtain an image quality score;

[0083] Based on the image quality score, several character images are repaired and combined to obtain a combined image;

[0084] Character recognition is performed on the combined image to obtain the object's inkjet code.

[0085] Based on this, the technical problems of existing technologies being unable to guarantee character integrity and lacking recognition robustness under conditions of strong reflection, blurring, or occlusion are solved.

[0086] like Figure 2 As shown in the embodiments of this application, the intelligent recognition method for steel plate inkjet printing based on multi-camera collaboration includes:

[0087] S201. Acquire several original images of the coding area on the same object using several image acquisition devices.

[0088] The coding area is a region that includes several characters.

[0089] For example, the image acquisition device can be a camera, video recorder, etc. Six 20-megapixel industrial cameras (model: Basler ace acA2440-75um) are arranged in a ring at 60-degree intervals 1.5 meters above the steel plate conveyor belt, and are simultaneously triggered to acquire the inkjet-printed area on the surface of a 12mm thick Q235B steel plate. Due to the difference in the viewing angle of each camera, six original 1920×1200 pixel images containing the inkjet-printed characters "CR28801A" are captured from 0° (front view), 60°, 120°, 180°, 240° and 300° viewing angles, respectively, with a uniform image frame rate of 75fps.

[0090] S202. Based on camera calibration, perform image alignment on the original image to obtain several aligned images.

[0091] It should be noted that the camera calibration is achieved by projecting images from cameras with different viewpoints onto a unified coordinate system using a calibration plate, thereby aligning the images.

[0092] For example, the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter rotation and translation matrix of the six cameras, which were obtained in advance using the 12×9 checkerboard calibration method, were used to calculate the sub-pixel precision mapping parameters using the Zhang Zhengyou calibration method. The six 1920×1200 pixel original images acquired by S201 were subjected to radial distortion correction and perspective transformation using the bilinear interpolation algorithm. The images with viewing angles from 60° to 300° were uniformly mapped to the 0° frontal camera coordinate system, so that the root mean square error (RMSE) of the stroke contour registration of the character "CR28801A" in all images was reduced to within 0.3 pixels. Finally, a pixel-level aligned six-view image dataset was generated.

[0093] S203. Detect and crop the characters in each aligned image to obtain several character images.

[0094] For example, a YOLOv7-tiny model with transfer learning (pre-trained on the MVTec character dataset, then fine-tuned with 5000 steel plate inkjet images, achieving 98.7% mAP@0.5 after 150 epochs) was used to perform inference detection on six 1920×1200 pixel aligned images generated by S202. Using a confidence threshold of 0.85 and an NMS threshold of 0.4, the bounding boxes of 8 characters in each image (character sequence "CR28801A") were accurately detected. Subsequently, each character was cropped according to the bounding box coordinates, and finally 48 grayscale character images with a size normalized to 80×80 pixels (saved as PNG format) were output, with an average size of 2.3KB for a single character image.

[0095] S204. Perform quality assessment on several character images to obtain image quality scores.

[0096] In some implementations, the quality assessment of several character images is performed, such as... Figure 3 As shown, it includes:

[0097] The character structure integrity of the character image is evaluated to obtain a character structure integrity score;

[0098] The reflectivity of characters in a character image is evaluated to obtain a character reflectivity score;

[0099] The blurriness of the characters in the image is evaluated to obtain a blurriness score.

[0100] The image quality score is calculated using an image quality formula; the expression for the image quality formula is:

[0101] ;

[0102] In the formula, For the normalized character structure integrity score, To normalize the character reflectivity score, The score represents the normalized character ambiguity level, with α, β, and γ being the weighting coefficients.

[0103] It should be noted that a higher image quality score Q indicates higher image quality. In practice, the three scores have different degrees of impact on character recognizability. Therefore, different weights are assigned using different weighting coefficients to evaluate the overall image quality.

[0104] In this embodiment, α is determined based on the complexity of the character's structure. For example, when there are many strokes, the range of α is larger. β is strongly correlated with the frequency of reflections in the production scene and the degree to which reflections obscure character details, i.e., the reflectivity of the object's surface. The more easily the material of the object reflects light and the more unstable the scene's light source, the larger β becomes. γ is strongly correlated with the frequency of blurring in the production scene and the degree to which blurring damages the character's edges. If the camera's focus is not stable enough, it can easily lead to blurred edges, thus the value of γ is larger.

[0105] For example, a quality assessment is performed on the character image "C": its structural integrity score is calculated. =0.92, reflectivity score =0.15, fuzziness score =0.08; set the weight coefficients α=0.7, β=0.2, γ=0.1, and substitute them into the image quality formula Q=(0.7×0.92) / (0.2×0.15+0.1×0.08)=0.644 / (0.03+0.008)=0.644 / 0.038≈16.95; finally, the Q value is mapped to the [0,1] interval by the Sigmoid function to obtain an image quality score of 0.94, indicating that the character image quality is excellent.

[0106] S205. Based on the image quality score, repair and combine several character images to obtain a combined image.

[0107] In some implementations, the step of repairing and combining several character images based on image quality scores includes:

[0108] The highest image quality score is selected for each character to obtain the highest quality score;

[0109] The highest quality score is compared with a preset threshold.

[0110] When the highest quality score is greater than or equal to a preset threshold, the character image corresponding to the highest quality score is marked as the final image;

[0111] When the highest quality score is less than the preset threshold, the character image corresponding to the highest quality score is repaired to obtain the repaired image, and the repaired image is marked as the final image;

[0112] Several final images are combined according to their coordinate positions to obtain a composite image.

[0113] It should be noted that when the image quality of a certain character is poor in all camera views, it needs to be repaired to improve the image quality. The repaired image is then combined with other high-quality character images in sequence.

[0114] For example, for the eight character positions in the string "CR28801A", the character image with the highest quality score for each character is selected from the 48 images evaluated in S204. The highest scores for the characters "2" and "A" are 0.79 and 0.81, respectively, which are lower than the preset threshold of 0.82. The images are repaired, and the quality scores of the repaired images are improved to 0.87 and 0.89. The highest scores of the remaining six characters (0.86-0.94) all exceed the threshold and are directly used as the final images. Finally, the eight 80×80 pixel images (including two repaired images) are horizontally combined in the original sequence order to generate a high-quality complete inkjet printing combination image of 675×80 pixels.

[0115] S206. Perform character recognition on the combined image to obtain the object's inkjet code.

[0116] For example, the PaddleOCR v2.3 recognition engine, deployed on an NVIDIA Tesla T4 GPU environment, was used to load the SVTR-Large model, which is optimized for steel plate inkjet printing (trained for 200 epochs on a dataset containing 500,000 steel plate characters, with a test set accuracy of 99.1%). The model was used to perform inference recognition on the 675×80 pixel combined image generated by S205. The confidence threshold was set to 0.92, the recognition time was 38ms, and the string "CR28801A" was accurately output (confidence 0.983). At the same time, the recognition timestamp, device number and quality evaluation score were recorded to the SQL Server database to complete the automated collection and traceability of steel plate inkjet printing.

[0117] Based on the above technical solution, the intelligent recognition method for steel plate inkjet printing based on multi-camera collaboration provided in this application effectively overcomes the problem of missing character features caused by reflection, blurring or occlusion in single-view imaging by using multi-view image collaborative acquisition and intelligent fusion: camera calibration and image alignment are used to ensure spatial consistency of multi-source data, and character-level quality assessment and optimal feature extraction mechanism are combined to accurately select the clearest and most complete area of ​​each character for dynamic repair and combination, which significantly improves the integrity and recognizability of the character area; finally, recognition is performed through high-quality combined images, which greatly enhances the anti-interference ability and recognition accuracy of the system in complex industrial environments and solves the problem of poor robustness of traditional methods.

[0118] In one possible implementation of this application embodiment, the above-mentioned S204 can be specifically implemented by the following S301, S302 and S303, which are described in detail below:

[0119] S301. Evaluate the character structure integrity of the character image and obtain a character structure integrity score.

[0120] In some implementations, the evaluation of the character structure integrity of the character image is performed, such as... Figure 4 As shown, it includes:

[0121] The skeleton of a character is extracted from a character image using a morphological thinning algorithm to obtain a character skeleton map;

[0122] Breakpoints are identified and counted in the character skeleton diagram to obtain the number of breakpoints;

[0123] The skeleton in the character skeleton diagram is decomposed into a point sequence by a tracking algorithm, and the approximate curvature of the point sequence is calculated to obtain a curvature sequence.

[0124] Calculate the standard deviation of the curvature sequence to obtain the degree of character deformation;

[0125] The number of breakpoints and the degree of character deformation are normalized and then weighted and summed to obtain the character structure integrity score.

[0126] It should be noted that the steps of decomposing the skeleton into a point sequence are as follows: select the starting endpoint of the skeleton, and start tracing the skeleton pixels using the Freeman chain code tracing algorithm from the starting endpoint until another endpoint is encountered to end the tracing. Store the pixel coordinates passed during the tracing process in a list to form an ordered point sequence. The degree of character deformation indicates the degree of deformation of the character. The greater the degree of character deformation, the more severe the skeleton jitter, the less smooth it is, the greater the difference from the standard form, and the worse the image quality.

[0127] For example, the structural integrity of the 80×80 pixel character image "2" is evaluated as follows: First, the Zhang-Suen thinning algorithm is used to extract the character skeleton, and three break points are found in the skeleton. Then, the Freeman chain code tracing algorithm is used to decompose the skeleton into a point sequence and calculate the approximate curvature, and the standard deviation of the curvature sequence is 0.17. The normalized value of the number of break points (3 / 10=0.3) and the normalized value of the standard deviation of curvature (0.17 / 0.5=0.34) are weighted and summed with weight coefficients of 0.6 and 0.4, respectively. Finally, the structural integrity score of the character is obtained as D_s=0.6×(1-0.3)+0.4×(1-0.34)=0.6×0.7+0.4×0.66=0.42+0.264=0.684.

[0128] S302. Evaluate the reflectivity of the characters in the character image and obtain a character reflectivity score.

[0129] In some implementations, the evaluation of the reflectivity of the characters in the character image includes:

[0130] Binarize the character image to obtain a specular mask image;

[0131] Perform a logical AND operation between the pixels of the specular mask image and the character skeleton image, and count the number of overlapping pixels;

[0132] Count the number of pixels in the skeleton region of the character skeleton image to obtain the total number of pixels;

[0133] The ratio of overlapping pixels to the total number of pixels is used to obtain a score for the character's reflectivity.

[0134] For example, the reflectivity of an 80×80 pixel character image "R" is evaluated as follows: First, the character image is binarized using a local adaptive thresholding method (window size 15×15, constant C=10) to generate a specular mask image (white areas are highlights); then, the specular mask image and the character skeleton image are logically ANDed, and the number of overlapping pixels is found to be 38; the total number of pixels in the character skeleton image is 210; the percentage of overlapping pixels is calculated to be 38 / 210≈0.181; the character reflectivity score D_r is this ratio itself (0.181), and a higher score indicates that the key structural areas of the character are more severely affected by highlights.

[0135] S303. Evaluate the blurriness of the characters in the character image and obtain a blurriness score.

[0136] In some implementations, the evaluation of the blurriness of the characters in the character image includes:

[0137] Perform a character masking operation on the character image to obtain the character mask region;

[0138] The sharpness response map is obtained by convolving the character image using a multi-directional kernel function.

[0139] Extract the response values ​​of the corresponding character mask regions from the sharpness response map to obtain the character response map;

[0140] Calculate the variance of the character response map to obtain the character ambiguity score.

[0141] It should be noted that only the blurriness of the character mask area is evaluated to avoid background interference, which allows for a more precise focus on the blurriness of the character portion, ensuring the focus and effectiveness of the evaluation.

[0142] For example, the blur level of an 80×80 pixel character image "0" is evaluated as follows: First, a binary image generated by the Otsu algorithm is used as a character mask to extract the character foreground region; then, a four-direction (0°, 45°, 90°, 135°) Sobel operator is used for convolution and the maximum response value is taken, followed by convolution with a Laplacian kernel (kernel size=3) to obtain a sharpness response map; the response values ​​of all pixels within the character mask region in the image are extracted, and its variance is calculated to be 152.8; the variance value is normalized to the [0,1] interval (divided by the preset maximum value of 1000) to obtain the character blur level score D_b=0.1528, the lower the value, the sharper the image.

[0143] Based on the above technical solution, a multi-dimensional quantitative evaluation system comprehensively and accurately characterizes the quality of character images: skeleton analysis effectively quantifies structural integrity, detecting stroke defects and deformations through breakpoints and deformation degrees; the fusion analysis of specular masks and skeletons accurately assesses the impact of reflections on key character structures; and multi-directional convolution and variance calculation objectively reflect edge sharpness. These three indicators are evaluated from three orthogonal dimensions: structural integrity, optical properties, and texture sharpness, overcoming the limitations of single indicators and providing reliable and interpretable quantitative basis for subsequent image screening and restoration, significantly improving the system's robustness in complex industrial environments.

[0144] In one possible implementation of this application embodiment, the above-mentioned S205 can be specifically implemented by the following S401, which will be described in detail below:

[0145] S401. Repair the character image corresponding to the highest quality score to obtain the repaired image.

[0146] In some implementations, the repair of the character image corresponding to the highest quality score includes:

[0147] Defect detection is performed on the character image to generate a mask of the defect region;

[0148] Calculate the image quality of the character image on the defect region mask to obtain the defect region quality score;

[0149] Calculate the image quality of the remaining character images in the defect area mask and select the maximum value to obtain the candidate image quality score;

[0150] The quality score of the defective region is compared with the quality score of the candidate image.

[0151] When the quality score of the defective region is greater than or equal to the quality score of the candidate image, the character image corresponding to the quality score of the defective region is marked as the repaired image.

[0152] When the quality score of the defective region is less than the quality score of the candidate image, the character image corresponding to the quality score of the candidate image is cropped according to the defective region mask to obtain the cropped region; the character image corresponding to the quality score of the defective region is replaced according to the cropped region to obtain the patch image, and the patch image is marked as the repaired image.

[0153] It should be noted that when repairing character images, the image with the best overall quality of the character is selected first. Then, the selected character image is stitched together with images of better parts of the character from other perspectives to achieve an overall improvement in the character image.

[0154] For example, a low-quality image of the character "2" (original quality score 0.79) is repaired as follows: First, the U-Net segmentation model identifies a defective region with high light pollution (the defect mask area accounts for 30% of the character region). The average Laplacian variance of this region in the original image is calculated to be 120 (defect region quality score). Then, five other viewpoint images of the same character position are traversed, and their Laplacian variances in the same defect mask region are calculated, with the maximum value of 285 taken (candidate image quality score). Since 120 < 285, the candidate image with a variance of 285 is selected, and the pixel blocks corresponding to its defect mask region are cropped out and seamlessly replaced with the corresponding region of the original low-quality image using the Poisson fusion algorithm to generate a patch image. After repair, the quality score of the character image is improved to 0.87 and is marked as the repaired image.

[0155] Based on the above technical solution, a multi-view image collaborative restoration mechanism significantly improves the final quality of character images. Its core effects are reflected in three aspects: First, by using a defect region mask, the location of image damage is accurately located, avoiding the waste of resources in global processing; second, intelligent comparison using a multi-view image library selects the optimal candidate region, ensuring the highest quality of the restored material; finally, a region replacement strategy achieves pixel-level precise restoration, preserving the background information of the original image while perfectly integrating high-quality character features. This adaptive restoration mechanism based on quality assessment effectively solves problems such as abnormal lighting, local blurring, or partial occlusion that may exist in a single image, providing highly complete input data for subsequent character recognition.

[0156] In one possible implementation of this application embodiment, the above-mentioned S302 can be specifically implemented by the following S701, which will be described in detail below:

[0157] S701. Perform binarization on the character image to obtain a specular mask image.

[0158] In some implementations, the binarization operation on the character image is performed, such as... Figure 5 As shown, it includes:

[0159] Set the window size N; where N is a positive integer;

[0160] The neighborhood weighted mean of each pixel in the character image is calculated based on the window size N, and the difference is taken with the preset constant C to obtain the initial threshold matrix;

[0161] By iterating through several character images and calculating the average of several thresholds corresponding to the same pixel coordinate position, a standard threshold matrix is ​​obtained.

[0162] The character image is converted into a binary image based on the standard threshold matrix and labeled as a specular mask image.

[0163] It should be noted that the average threshold of several images from different viewpoints corresponding to the same pixel coordinate position is to construct a stable and universal standard threshold template, so as to eliminate the local threshold deviation of a single image caused by factors such as shooting viewpoint and lighting conditions, thereby significantly improving the robustness and consistency of the binarization results.

[0164] For example, with a window size of N=15, firstly, for each pixel in the input character image, calculate the Gaussian weighted mean (standard deviation of 7) of the pixels in its 15×15 neighborhood, and subtract the preset constant C=10 from this mean to obtain the initial threshold matrix; then, iterate through the initial thresholds corresponding to the same pixel coordinate position in 6 character images of the same character, and calculate the arithmetic mean to generate the standard threshold matrix; finally, compare each pixel value of the character image to be processed with the threshold at the corresponding position in the standard threshold matrix: if the pixel value is greater than the threshold, set it to 255 (white), otherwise set it to 0 (black). The white area in the generated specular mask image represents the specular feature region.

[0165] Based on the above technical solution, the problem of single-image binarization being sensitive to illumination is effectively solved by combining local adaptive thresholding with multi-image statistics: Gaussian weighted mean calculation eliminates local noise interference, and the adjustment of the constant C enhances the sensitivity to highlight areas; the standard threshold matrix constructed by multi-image threshold averaging significantly improves the robustness and generalization ability of highlight detection, and the final generated highlight mask can accurately identify reflective areas under different illumination conditions, providing a reliable basis for subsequent character quality assessment.

[0166] In one possible implementation of this application embodiment, the above-mentioned S303 can be specifically implemented by the following S801, which will be described in detail below:

[0167] S801. Convolution calculation is performed on the character image using a multi-directional kernel function to obtain a sharpness response map.

[0168] In some implementations, the convolution calculation of the character image using a multi-directional kernel function includes:

[0169] Several gradient operators in different directions are used to convolve with the character image to obtain several gradient response maps;

[0170] The maximum response value of each pixel in several gradient response maps is selected to form a maximum response map.

[0171] The sharpness response map is obtained by performing a second convolution on the maximum response map using a Laplacian kernel.

[0172] For example, Sobel operators (kernel size 3×3, x-direction kernel [-1,0,1; -2,0,2; -1,0,1], y-direction kernel [-1,-2,-1; 0,0,0; 1,2,1]) in four directions (0°, 45°, 90°, and 135°) are convolved with an 80×80 pixel grayscale character image (pixel value range 0-255) to generate four gradient response maps (data type float32, value range approximately [-1020, 1020]). The four response maps are compared pixel by pixel, and the maximum value is selected to generate a maximum response map (average pixel value approximately 186.5). Then, a second convolution is performed using a Laplacian kernel (kernel [[0,1,0],[1,-4,1],[0,1,0]], standardization factor 1), finally obtaining a sharpness response map with a mean variance of 285.3 and a standard deviation of approximately 16.89.

[0173] Based on the above technical solution, multi-directional gradient operator convolution effectively captures edge information in all directions of the character, and uses the strategy of taking the maximum value to enhance these edge features, generating a maximum response map with enhanced edges. Subsequently, a Laplacian kernel is introduced for secondary convolution, and its second-order differential property further sharpens the edges and highlights the details and textures. The final sharpness response map significantly improves the overall contrast and detail resolution of the image, providing a highly discriminative feature data foundation for character blur assessment.

[0174] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as an electronic device, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0176] When using integrated units, Figure 6 A possible structural schematic diagram of the electronic device (denoted as electronic device 50) involved in the above embodiments is shown. The electronic device 50 includes a processing unit 501 and a communication unit 502, and may also include a storage unit 503. Figure 6 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0177] when Figure 6 The schematic diagram shown is used to illustrate the structure of the electronic device involved in the above embodiments. The processing unit 501 is used to control and manage the operation of the electronic device, the communication unit 502 is used for the electronic device to communicate with other devices, and the storage unit 503 is used to store the program code and data of the electronic device.

[0178] For example, communication unit 502 is used to acquire several original images of the coding area on the same object through several image acquisition devices;

[0179] The processing unit 501 is used to perform image alignment on the original image based on camera calibration to obtain several aligned images; to detect and crop characters in each aligned image to obtain several character images; to perform quality evaluation on the several character images to obtain image quality scores; to repair and combine the several character images according to the image quality scores to obtain a combined image; and to perform character recognition on the combined image to obtain object inkjet printing.

[0180] The processing unit 501 can be a processor or a controller, and the communication unit 502 can be a communication interface, transceiver, transceiver circuit, transceiver device, etc. The term "communication interface" is a general term and may include one or more interfaces. The storage unit 503 can be a memory. When the electronic device 50 is a chip, the processing unit 501 can be a processor or a controller, and the communication unit 502 can be an input interface and / or an output interface, pins, or circuits, etc. The storage unit 503 can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip (e.g., read-only memory (ROM), random access memory (RAM, etc.).

[0181] The communication unit can also be called a transceiver unit. The antenna and control circuit with transceiver functions in the electronic device 50 can be considered as the communication unit 502 of the electronic device 50, and the processor with processing functions can be considered as the processing unit 501 of the electronic device 50. Optionally, the device in the communication unit 502 used to implement the receiving function can be considered as the communication unit. The communication unit is used to execute the receiving steps in the embodiments of this application, and the communication unit can be a receiver, a receiver circuit, etc. The device in the communication unit 502 used to implement the transmitting function can be considered as the transmitting unit. The transmitting unit is used to execute the transmitting steps in the embodiments of this application, and the transmitting unit can be a transmitter, a transmitter, a transmitting circuit, etc.

[0182] Figure 6 If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0183] Figure 6 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0184] This application also provides a hardware structure diagram of an electronic device (denoted as electronic device 60), see [link to diagram]. Figure 7 The electronic device 60 includes a processor 601, and optionally, a memory 602 connected to the processor 601.

[0185] In the first possible implementation, see Figure 7 The electronic device 60 also includes a transceiver 603. The processor 601, memory 602, and transceiver 603 are connected via a bus. The transceiver 603 is used to communicate with other devices or communication networks. Optionally, the transceiver 603 may include a transmitter and a receiver. The device in the transceiver 603 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 603 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0186] Based on the first possible implementation method Figure 7 The structural diagram shown can be used to illustrate the structure of the electronic device involved in the above embodiments.

[0187] in, Figure 7 This can also be illustrated by a system chip in an electronic device. In this case, the actions performed by the aforementioned electronic device can be implemented by this system chip; the specific actions performed can be found above and will not be repeated here.

[0188] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0189] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a separate semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (System-on-a-Chip), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0190] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0191] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0192] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0193] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0194] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0195] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0196] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for intelligent recognition of steel plate inkjet printing based on multi-camera collaboration, characterized in that, include: Several original images of a coding area on the same object are acquired using several image acquisition devices; wherein, the coding area is an area that includes several characters; Based on camera calibration, the original images are aligned to obtain several aligned images; The characters in each aligned image are detected and cropped to obtain several character images; A quality assessment is performed on several character images to obtain an image quality score; Based on the image quality score, several character images are repaired and combined to obtain a combined image; Character recognition is performed on the combined image to obtain the object's inkjet code; The process of repairing and combining several character images based on image quality scores includes: The highest image quality score is selected for each character to obtain the highest quality score; The highest quality score is compared with a preset threshold. When the highest quality score is greater than or equal to a preset threshold, the character image corresponding to the highest quality score is marked as the final image; When the highest quality score is less than the preset threshold, the character image corresponding to the highest quality score is repaired to obtain the repaired image, and the repaired image is marked as the final image; Several final images are combined according to their coordinate positions to obtain a composite image; The restoration of the character image corresponding to the highest quality score includes: Defect detection is performed on the character image to generate a mask of the defect region; Calculate the image quality of the character image on the defect region mask to obtain the defect region quality score; Calculate the image quality of the remaining character images in the defect area mask and select the maximum value to obtain the candidate image quality score; The quality score of the defective region is compared with the quality score of the candidate image. When the quality score of the defective region is greater than or equal to the quality score of the candidate image, the character image corresponding to the quality score of the defective region is marked as the repaired image. When the quality score of the defective region is less than the quality score of the candidate image, the character image corresponding to the quality score of the candidate image is cropped according to the defective region mask to obtain the cropped region; the character image corresponding to the quality score of the defective region is replaced according to the cropped region to obtain the patch image, and the patch image is marked as the repaired image.

2. The method according to claim 1, characterized in that, The quality assessment of several character images includes: The character structure integrity of the character image is evaluated to obtain a character structure integrity score; The reflectivity of characters in a character image is evaluated to obtain a character reflectivity score; The blurriness of the characters in the image is evaluated to obtain a blurriness score. The image quality score is calculated using an image quality formula; the expression for the image quality formula is: ； In the formula, For the normalized character structure integrity score, To normalize the character reflectivity score, The score represents the normalized character ambiguity level, with α, β, and γ being the weighting coefficients.

3. The method according to claim 2, characterized in that, The evaluation of the character structure integrity of the character image includes: The skeleton of a character is extracted from a character image using a morphological thinning algorithm to obtain a character skeleton map; Breakpoints are identified and counted in the character skeleton diagram to obtain the number of breakpoints; The skeleton in the character skeleton diagram is decomposed into a point sequence by a tracking algorithm, and the approximate curvature of the point sequence is calculated to obtain a curvature sequence. Calculate the standard deviation of the curvature sequence to obtain the degree of character deformation; The number of breakpoints and the degree of character deformation are normalized and then weighted and summed to obtain the character structure integrity score.

4. The method according to claim 3, characterized in that, The evaluation of the reflectivity of characters in the character image includes: Binarize the character image to obtain a specular mask image; Perform a logical AND operation between the pixels of the specular mask image and the character skeleton image, and count the number of overlapping pixels; Count the number of pixels in the skeleton region of the character skeleton image to obtain the total number of pixels; The ratio of overlapping pixels to the total number of pixels is used to obtain a score for the character's reflectivity.

5. The method according to claim 4, characterized in that, The binarization operation on the character image includes: Set the window size N; where N is a positive integer; The neighborhood weighted mean of each pixel in the character image is calculated based on the window size N, and the difference is taken with the preset constant C to obtain the initial threshold matrix; By iterating through several character images and calculating the average of several thresholds corresponding to the same pixel coordinate position, a standard threshold matrix is ​​obtained. The character image is converted into a binary image based on the standard threshold matrix and labeled as a specular mask image.

6. The method according to claim 2, characterized in that, The evaluation of the blurriness of characters in the character image includes: Perform a character masking operation on the character image to obtain the character mask region; The sharpness response map is obtained by convolving the character image using a multi-directional kernel function. Extract the response values ​​of the corresponding character mask regions from the sharpness response map to obtain the character response map; Calculate the variance of the character response map to obtain the character ambiguity score.

7. The method according to claim 6, characterized in that, The method of performing convolution calculations on character images using multi-directional kernel functions includes: Several gradient operators in different directions are used to convolve with the character image to obtain several gradient response maps; The maximum response value of each pixel in several gradient response maps is selected to form a maximum response map. The sharpness response map is obtained by performing a second convolution on the maximum response map using a Laplacian kernel.

8. An electronic device, used in the method as described in any one of claims 1-7, characterized in that, include: Communication unit and processing unit; The communication unit is used to acquire several original images of the inkjet printing area on the same object through several image acquisition devices; The processing unit is used to perform image alignment on the original image based on camera calibration to obtain several aligned images; to detect and crop characters in each aligned image to obtain several character images; to perform quality evaluation on the several character images to obtain image quality scores; to repair and combine the several character images according to the image quality scores to obtain combined images; and to perform character recognition on the combined images to obtain object inkjet codes.

Images