A printing defect detection method and device based on cross-modal alignment

Abstract

The application discloses a printing defect detection method and device based on cross-modal alignment, relates to the technical field of printing defect detection, and realizes cross-modal feature alignment of a first printing target image (a color image) and a second printing target image (a gray-scale image), extracts geometric transformation parameters of the two images to complete pixel-level registration; converts the registered image into a gray-scale image and performs channel dimension splicing to generate a spliced image; an anomaly detection algorithm is used to detect the spliced image, and the positioning of a printing defect is realized. The application breaks through the dependence of a traditional method on a reference image, and only needs a non-defect sample to complete model training, effectively solves the false detection problem caused by a complex texture background, an anti-counterfeiting film interference and personalized features in a small sample scene, and the time consumption of single detection is less than 0.3 seconds, which is more than 10 times higher than the efficiency of traditional manual detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of printing defect detection, and in particular to a printing defect detection method and device based on cross-modal alignment. BACKGROUND

[0002] In the card making process, image printing is an essential part. Image printing is divided into color image printing and gray image printing. Taking a card as an example, it is usually necessary to print the personalized information of the card owner, such as the name, card number and head-shoulder image. In order to prevent the card from being counterfeited, on the one hand, the printing of the head-shoulder image will use color printing and gray printing at the same time to facilitate double verification, and on the other hand, before printing the personalized information, a color background pattern with complex texture will be printed on the card, and after printing the personalized information, an anti-counterfeiting film with an anti-counterfeiting pattern will also be added. The color head-shoulder image is used for quick visual identification, has a large size, and the color is closer to the real photo, and is less disturbed by the color background pattern; while the gray head-shoulder image is used for machine identification, has a small size, and is easily disturbed by the color background pattern and the anti-counterfeiting film. During the printing of the head-shoulder image, equipment failure can cause various printing defects, including color head-shoulder image printing loss, printing color deviation and gray head-shoulder image printing loss. In order to ensure the quality of the card head-shoulder image printing, manual visual inspection is usually required. The traditional quality control method relying on manual visual inspection is not only low in efficiency (the average single detection time is more than 3 seconds), but also has the risk of missing detection. In order to realize efficient and accurate quality detection, automatic detection technology based on computer vision becomes an inevitable choice, but it needs to meet two core requirements: (1) the algorithm must have multi-defect collaborative detection capability, including but not limited to the above-mentioned printing defect types; (2) the printed image has personalized features (such as personalized portraits, etc.), and the algorithm needs to remain robust when printing personalized information.

[0003] The existing automatic detection technology usually relies on reference images to detect printing defects by comparing the differences between the reference images and the printed images. When there is no reference image, defect detection cannot be performed. The present application is directed to the relationship between the first printed target image and the second printed target image (both the first printed target image and the second printed target image come from the same electronic printing original, and the difference lies in the different inks used for printing, the different positions printed and the different scales), and based on the fact that the same type of defects with the same shape will not appear at the same position, the first printed target image and the second printed target image are used for mutual verification, thereby realizing automatic detection.

[0004] The implementation of this technology mainly faces the following technical challenges:

[0005] Firstly, when printing an image, the surface of the printing substrate has already been printed with a complex texture pattern. When printing the image on the surface of the printing substrate according to the specified position, due to equipment errors, the actual printing position deviates from the specified position. The complex texture pattern on the surface of the printing substrate interferes with the printed image, and the interference caused by the complex texture pattern is unpredictable due to the unpredictability of the printing position. In the post-processing link, a layer of anti-counterfeiting film is often added, and the anti-counterfeiting film usually also contains a complex anti-counterfeiting pattern, which interferes with the imaging, and the position of the anti-counterfeiting film also deviates, so that the interference caused by the anti-counterfeiting pattern on the anti-counterfeiting film to the printed image also has unpredictability. The above deviations result in the need to simultaneously process the unpredictable interference caused by the complex texture pattern on the surface of the printing substrate and the anti-counterfeiting film when performing printing defect detection.

[0006] Secondly, there are multiple interference factors in the imaging link: the color distortion generated in the printing process and the imaging environment light interference superimpose to form a complex color deviation, which makes it difficult for traditional color difference determination methods to accurately identify the true color difference. At the same time, the complex background pattern on the surface of the printing substrate and the high-reflectivity characteristics of the post-processing anti-counterfeiting film easily cause the dynamic range compression of the imaging system (typical performance is the loss of details in the highlight area), causing false defect interference misjudgment.

[0007] Thirdly, there are algorithm challenges in the defect identification level: the uncertainty of defect morphology and position and the unpredictability of individualized image features restrict the generalization ability of traditional supervised learning models.

[0008] Finally, there is a data bottleneck in model training: the number of defect samples collected in actual applications is limited, which makes the traditional supervised learning model prone to overfitting and missed detection. For example, the defect sample occurrence probability of the gray printed portrait of the card certificate is less than one in a thousand, which is difficult to support supervised learning. Correspondingly, a large number of non-defect samples can be collected and applied to model training.

[0009] In view of the above technical bottlenecks, it is urgent to develop an intelligent high-precision detection solution to realize real-time and accurate identification of image printing defects under complex working conditions. SUMMARY

[0010] In order to solve the above technical defects, the present application provides a printing defect detection method and device based on cross-modal alignment. The following technical solutions are adopted:

[0011] A printing defect detection method based on cross-modal alignment, comprising the following steps:

[0012] Step 1: acquiring a color printed image as a first detection image through a visible light camera, and acquiring a gray printed image as a second detection image through an infrared camera;

[0013] Step 2, detecting the color printing result as a first printing target in the first to-be-detected image and detecting the gray printing result as a second printing target in the second to-be-detected image;

[0014] Step 3, aligning the first printing target and the second printing target to obtain a first aligned image and a second aligned image;

[0015] Step 4, constructing a defect-sensitive feature space, obtaining a gray image of the first aligned image and the second aligned image, and splicing according to the channel to obtain a spliced image;

[0016] Step 5, training an anomaly detector using the spliced image of the defect-free sample, and inputting the spliced image into the anomaly detector for anomaly detection;

[0017] Step 6, combining the a channel feature of the Lab color space on the spliced image to distinguish color or gray printing defects, and realizing multi-defect collaborative detection.

[0018] By adopting the above technical solution, the color printing image (the first to-be-detected image) is obtained by the visible light camera in the controllable environment, the gray printing image (the second to-be-detected image) is obtained by the infrared camera, and the background interference is suppressed by using the ink spectral characteristic difference. The inks used in color printing and gray printing are different, in the infrared camera, the high light interference of the color background and the anti-counterfeiting film is suppressed by using the low reflection characteristic of the gray printing ink. The ink of the color printing has a high reflection characteristic in the infrared camera, and the reflection degree hardly changes with the color of the printing ink, while the ink of the gray printing has a low reflection characteristic. By using the infrared camera imaging, the interference of the complex background pattern and the anti-counterfeiting film of the color printing on the gray printing target can be greatly reduced. The color printing result of the image is imaged under the visible light camera, and the target presents color. The gray printing result of the image is imaged under the infrared camera, and the color background pattern and the anti-counterfeiting film show a consistent high brightness, while the gray printing ink shows a low brightness, which can clearly distinguish the printed object from the background. On the one hand, in the controllable environment, the use of light source is controlled to avoid color distortion of the imaging of the visible light camera, and on the other hand, the infrared camera can reduce the interference of the complex background and the anti-counterfeiting film on the gray printing result.

[0019] By realizing the cross-modal feature alignment of the first printing target image (color image) and the second printing target image (gray image), the geometric transformation parameters of the two images are extracted to complete the pixel-level registration; the registered image is converted into a gray image and spliced in the channel dimension to generate a spliced image; an anomaly detection algorithm is used to detect the spliced image, and the positioning of the printing defects is realized.

[0020] The method breaks through the dependence of traditional methods on reference images, and can complete model training only by using a defect-free sample, effectively solving the false detection problem caused by complex texture background, anti-counterfeiting film interference and personalized features in a small sample scene. The single-piece detection time is less than 0.3 seconds, which is more than 10 times the efficiency of traditional manual detection.

[0021] Optionally, in step 2, a target detector is trained based on YOLOv5, and the first printed target is detected in the first to-be-detected image and the second printed target is detected in the second to-be-detected image through target detection.

[0022] Optionally, the specific method of step 3 is: adjusting the scale of the first printed target according to the scale ratio of the first printed target and the second printed target, so that the adjusted first printed target and the second printed target have the same scale, and registering the first printed target and the second printed target by using an ORB feature point matching algorithm. The registered images are a first aligned image and a second aligned image, respectively.

[0023] By adopting the above technical solution, cross-modal alignment of the first printed target and the second printed target at the pixel level can be realized.

[0024] Optionally, the method for obtaining the spliced image in step 4 is: taking the gray image of the first aligned image as a reference, performing gray normalization on the second aligned image, so that the pixel mean value of the second aligned image is equal to the pixel mean value of the gray image of the first aligned image, to obtain the second aligned image after gray normalization; taking the gray image of the first aligned image as the blue channel of the spliced image, taking the second aligned image after gray normalization as the green channel of the spliced image, and setting the red channel of the spliced image to zero.

[0025] By adopting the above technical solution, the gray images of the first aligned image and the second aligned image are obtained, and the spliced image is obtained by splicing according to the channels. The specific implementation steps are: mapping the color printing gray image to the blue channel (B), mapping the gray printing gray image to the green channel (G), and setting the red channel (R) to zero, thereby constructing a three-dimensional feature space. In this feature space, the defect representation presents a significant color differentiation characteristic: when the color printing has missing printing or color deviation, the B channel is activated to cause the feature vector to present a blue response; when the gray printing has missing printing defects, the G channel is activated to present a green feature; under normal working conditions, the dark area (such as black hair and dark clothing in the head and shoulder image) presents a black feature because the gray values of the two channels tend to zero, and the light area (such as skin and light clothing) presents a cyan representation because the high gray values of the two channels are superimposed. This method realizes explicit decoupling of defect features by constructing an orthogonal color space, effectively improving the feature separability. Compared with the traditional single-channel detection method, the feature space has the following advantages:

[0026] 1) The multi-modal detection problem is converted into a pattern recognition problem in the feature space by a color coding mechanism, which significantly reduces the complexity of the detection model.

[0027] 2) Based on the feature distribution modeling of normal samples, the overfitting problem caused by personalized images (such as diversified head and shoulder features) can be effectively overcome by using an unsupervised learning strategy.

[0028] Optionally, in step 5, the anomaly detector implements anomaly detection by using the anomaly detection algorithm Padim.

[0029] By using the above technical solution, the anomaly detection algorithm Padim is used, the feature representation of normal samples is learned, and a statistical model is used to capture the spatial distribution of these features, thereby realizing effective identification of abnormal regions. The anomaly detection method only uses defect-free samples to complete the training, without the need to collect a large number of defect samples.

[0030] Optionally, the backbone network of the Padim feature extraction network uses a ResNet network, and is pre-trained on a large-scale data color classification task, so that the obtained network parameters are more focused on color feature extraction.

[0031] Optionally, the threshold of the anomaly detector is set to 0.5, and the confidence map output by the Padim is binarized and morphologically opened to obtain the anomaly detection result.

[0032] By using the above technical solution, for the sample to be detected, the Padim first extracts high-dimensional features based on the pre-trained network, and then determines the abnormality of the sample to be detected according to the spatial distribution parameters of the normal samples, and outputs the corresponding confidence map, which reflects the abnormality degree. The backbone network of the feature extraction network is pre-trained on a large-scale data color classification task, and the obtained network parameters are more focused on color feature extraction. Since the defects have significant color distinguishing characteristics in the splicing graph, the feature extraction network focusing on color feature extraction can obtain better anomaly detection results. The confidence map output by the Padim is binarized and morphologically opened to obtain the anomaly detection result. The morphological opening operation can filter some small false positives.

[0033] Optionally, in step 6, the splicing graph is converted to Lab space, the a channel is taken, and it is judged whether the defect position is green. If it is green, it is a defect of the gray printing result, otherwise it is a defect of the color printing result.

[0034] Optionally, if a < 0, the defect position is judged to be green.

[0035] According to the generation method of the splicing image: taking the gray image of the first aligned image as a reference, the second aligned image is subjected to gray normalization, so that the pixel mean value of the second aligned image is equal to the pixel mean value of the gray image of the first aligned image, to obtain the second aligned image after gray normalization; taking the gray image of the first aligned image as the blue channel of the splicing image, and taking the second aligned image after gray normalization as the green channel of the splicing image, and setting the red channel of the splicing image to zero, if the gray printing has defects, for example, printing missing, then the green channel of the splicing image presents high brightness of the background image, that is, the green channel of the splicing image presents high brightness, thereby causing the splicing image to present green; on the contrary, if the color printing has defects, for example, printing missing, then the blue channel of the splicing image presents high brightness of the background image, that is, the blue channel of the splicing image presents high brightness, thereby causing the splicing image to present blue. In the Lab space, the a channel presents the component from green to red, and a<0, then the defect position is green, indicating that the gray printing result has defects, and on the contrary, indicating that the color printing has defects, by adopting the technical scheme, it can be determined according to the color condition of the splicing image and the abnormal detection result that the position of the defect is on the color printing target or on the gray printing target.

[0036] The memory stores a printing defect detection program designed by using the printing defect detection method based on cross-modal alignment.

[0037] The printing defect detection device based on cross-modal alignment comprises an image acquisition device, a memory, a processor and a display, the image acquisition device comprises a visible light camera and an infrared camera, the visible light camera is used for acquiring a first detection image, the infrared camera is used for acquiring a second detection image, the memory is in communication connection with the visible light camera and the infrared camera respectively, the memory stores a printing defect detection program designed by using the printing defect detection method based on cross-modal alignment, the processor is in communication connection with the memory, and the printing defect detection program is run to output a printing defect detection result, and the display is in communication connection with the processor, and the processor controls the display to display the printing defect detection result.

[0038] In summary, the present application has the following at least beneficial technical effects:

[0039] The application can provide a printing defect detection method and device based on cross-modal alignment. By realizing cross-modal feature alignment of a first printing target image (color image) and a second printing target image (gray image), geometric transformation parameters of the two images are extracted to complete pixel-level registration. The registered images are converted into gray images and channel dimension splicing is performed to generate a splicing image. An anomaly detection algorithm is used to detect the splicing image to realize the positioning of printing defects. The application breaks through the dependence of traditional methods on reference images, and only needs a defect-free sample to complete model training, effectively solving the false detection problem caused by complex texture background, anti-counterfeiting film interference and personalized features in a small sample scene. The single-piece detection time is less than 0.3 seconds, which is more than 10 times the efficiency of traditional manual detection. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flowchart of a printing defect detection method based on cross-modal alignment of the application; DETAILED DESCRIPTION

[0041] The application will be further described in detail below with reference to the accompanying drawings.

[0042] Embodiments of the application disclose a printing defect detection method and device based on cross-modal alignment.

[0043] REFERENCE Figure 1 , Embodiment 1, a printing defect detection method based on cross-modal alignment, comprising the following steps:

[0044] Step 1: acquiring a color printing image as a first detection image through a visible light camera, and acquiring a gray printing image as a second detection image through an infrared camera;

[0045] Step 2: detecting a color printing result as a first printing target in the first detection image, and detecting a gray printing result as a second printing target in the second detection image;

[0046] Step 3: aligning the first printing target and the second printing target to obtain a first aligned image and a second aligned image;

[0047] Step 4: constructing a defect-sensitive feature space, acquiring gray images of the first aligned image and the second aligned image, and splicing according to the channel to obtain a splicing image;

[0048] Step 5: training an anomaly detector using the splicing image of the defect-free sample, and inputting the splicing image into the anomaly detector for anomaly detection;

[0049] Step 6: combining the a channel feature of the Lab color space to distinguish color or gray printing defects on the splicing image according to the anomaly detection result, and realizing multi-defect collaborative detection.

[0050] By adopting the technical scheme, the color printed image (first to-be-detected image) is acquired by the visible light camera in a controllable environment, the gray-scale printed image (second to-be-detected image) is acquired by the infrared camera, and the background interference is suppressed by using the ink spectral characteristic difference. The inks used in color printing and gray-scale printing are different. In the infrared camera, the high light interference of the color background and the anti-counterfeiting film is suppressed by using the low reflection characteristic of the gray-scale printed ink. The ink of the color printed image has a high reflection characteristic in the infrared camera, and the reflection degree almost does not change with the color of the printed ink, while the ink of the gray-scale printed image has a low reflection characteristic. By imaging with the infrared camera, the interference of the complex background pattern of the color printed image and the anti-counterfeiting film on the gray-scale printed target can be greatly reduced. The color printed result of the image is imaged under the visible light camera, and the target presents color. The color printed result of the image is imaged under the infrared camera, the color background pattern and the anti-counterfeiting film show a relatively high brightness, while the gray-scale printed ink shows a relatively low brightness, so that the printed object and the background can be distinguished. On the one hand, in the controllable environment, the use of the light source is controlled to avoid color distortion of the imaging of the visible light camera, and on the other hand, the infrared camera can reduce the interference of the complex background and the anti-counterfeiting film on the gray-scale printed result.

[0051] By realizing the cross-modal feature alignment of the first printed target image (color image) and the second printed target image (gray-scale image), the geometric transformation parameters of the two images are extracted to complete pixel-level registration; the registered images are converted into gray-scale images and are spliced in the channel dimension to generate a spliced image; the spliced image is detected by using an anomaly detection algorithm to realize the positioning of the printing defects.

[0052] The traditional method is broken through the dependence on the reference image, and the model training can be completed only by using a defect-free sample, so that the false detection problem caused by the complex texture background, the anti-counterfeiting film interference and the personalized features in the small sample scene is effectively solved, the time consumption of single detection is less than 0.3 seconds, and the efficiency is more than 10 times higher than that of the traditional manual detection.

[0053] In step 2 of embodiment 2, a target detector is trained based on YOLOv5, the first printed target is detected in the first to-be-detected image by target detection, and the second printed target is detected in the second to-be-detected image.

[0054] In step 3 of embodiment 3, the specific method is as follows: according to the scale ratio of the first printed target and the second printed target, the scale of the first printed target is adjusted, so that the adjusted first printed target and the second printed target have the same scale, and the first printed target and the second printed target are registered by using the ORB feature point matching algorithm. The registered images are a first aligned image and a second aligned image.

[0055] By adopting the technical scheme, cross-modal alignment of the first printing target and the second printing target at a pixel level can be realized. In addition to the ORB feature point matching algorithm, SIFT or SURF feature point matching algorithm can also be adopted.

[0056] In step 4 of embodiment 4, the method for obtaining the spliced image is: taking the gray image of the first aligned image as a reference, performing gray normalization on the second aligned image, so that the pixel mean value of the second aligned image is equal to the pixel mean value of the gray image of the first aligned image, to obtain the second aligned image after gray normalization; taking the gray image of the first aligned image as the blue channel of the spliced image, taking the second aligned image after gray normalization as the green channel of the spliced image, and setting the red channel of the spliced image to zero.

[0057] The gray images of the first aligned image and the second aligned image are obtained, and are spliced according to the channels to obtain a spliced image. The specific implementation steps are: mapping the color printing gray image to the blue channel (B), mapping the gray printing gray image to the green channel (G), and setting the red channel (R) to zero, thereby constructing a three-dimensional feature space. In this feature space, the defect representation presents a significant color differentiation characteristic: when the color printing has missing printing or color deviation, the B channel is activated to cause the feature vector to present a blue response; when the gray printing has a missing printing defect, the G channel is activated to present a green feature; under normal working conditions of the double system, the dark color area (such as black hair and dark clothing in the head and shoulder image) presents a black feature because the gray values of the two channels tend to zero, and the light color area (such as skin and light clothing) presents a cyan representation because the high gray values of the two channels are superimposed. This method realizes explicit decoupling of the defect feature by constructing an orthogonal color space, effectively improving the feature separability. Compared with the traditional single-channel detection method, the feature space has the following advantages:

[0058] 1) The multi-modal detection problem is converted into a pattern recognition problem in the feature space through a color coding mechanism, which significantly reduces the complexity of the detection model;

[0059] 2) Based on the feature distribution modeling of normal samples, the non-supervised learning strategy can effectively overcome the overfitting problem caused by personalized images (such as diversified head and shoulder features).

[0060] In step 5 of embodiment 5, the anomaly detector adopts the anomaly detection algorithm Padim to realize anomaly detection.

[0061] The anomaly detection algorithm Padim is adopted. The anomaly detection method only uses defect-free samples for training, without the need to collect a large number of defect samples.

[0062] In embodiment 6, the threshold of the anomaly detector is set to 0.5, and the confidence map output by Padim is binarized and processed by morphological opening operation, to obtain the anomaly detection result.

[0063] By adopting the technical scheme, for the to-be-detected sample, the Padim first extracts high-dimensional features based on a pre-trained network, then determines the abnormality of the to-be-detected sample according to the spatial distribution parameters of the normal sample, and outputs a corresponding confidence map, which reflects the abnormality degree. The confidence map is binarized and subjected to a morphological opening operation, and the abnormality detection result can be obtained. The morphological opening operation can filter some small false positives.

[0064] In step 6 of embodiment 7, the spliced image is subjected to color space conversion to Lab space, the a channel is taken, and it is judged whether the defect position is green. If it is green, it is a defect of the gray printing result, otherwise it is a defect of the color printing result.

[0065] In embodiment 8, if a < 0, the defect position is judged to be green.

[0066] According to the generation method of the spliced image: taking the gray-scale image of the first aligned image as the reference, the gray-scale normalization is performed on the second aligned image, so that the pixel mean value of the second aligned image is equal to the pixel mean value of the gray-scale image of the first aligned image, to obtain the gray-scale normalized second aligned image; taking the gray-scale image of the first aligned image as the blue channel of the spliced image, and taking the gray-scale normalized second aligned image as the green channel of the spliced image, and setting the red channel of the spliced image to zero, if the gray printing has a defect, for example, printing missing, then the green channel of the spliced image presents the high brightness of the background image, that is, the green channel of the spliced image presents high brightness, so that the spliced image presents green; on the contrary, if the color printing has a defect, for example, printing missing, then the blue channel of the spliced image presents the high brightness of the background image, that is, the blue channel of the spliced image presents high brightness, so that the spliced image presents blue. In the Lab space, the a channel presents the component from green to red, a < 0, the defect position is green, indicating that the gray printing result has a defect, otherwise indicating that the color printing has a defect. By adopting the technical scheme, it can be determined whether the defect position is on the color printing target or on the gray printing target according to the color condition of the spliced image and the abnormality detection result.

[0067] In embodiment 9, a memory stores a printing defect detection program designed based on a cross-modal alignment-based printing defect detection method.

[0068] Embodiment 10, a printing defect detection device based on cross-modal alignment, comprising an image acquisition device, a memory, a processor and a display, the image acquisition device comprises a visible light camera and an infrared camera, the visible light camera is used to shoot and acquire a first detection image, the infrared camera is used to shoot and acquire a second detection image, the memory is in communication connection with the visible light camera and the infrared camera respectively, the memory stores a printing defect detection program designed by using a printing defect detection method based on cross-modal alignment, the processor is in communication connection with the memory, runs the printing defect detection program to output a printing defect detection result, and the display is in communication connection with the processor, and the processor controls the display to display the printing defect detection result.

[0069] The following uses specific embodiments to illustrate the implementation principle of the printing defect detection method and device based on cross-modal alignment:

[0070] Taking the head and shoulder image in the card as an example, the first printing target is the color head and shoulder image, and the second printing target is the reduced head and shoulder image printed in grayscale. The printing defect detection is carried out according to the following steps, and whether the two printing targets have defects is detected, and the defect position is given.

[0071] Step 1, acquire a color printing image as a first detection image through a visible light camera, and acquire a grayscale printing image as a second detection image through an infrared camera;

[0072] Step 2, detect the color printing result in the first detection image as a first printing target, and detect the grayscale printing result in the second detection image as a second printing target; here, the YOLO detection algorithm can be trained and used for detecting the first printing target and the second printing target respectively. By using Tesla T4 graphics card, the target time consumption can be reduced to within 20 milliseconds.

[0073] Step 3, align the first printing target and the second printing target to obtain a first aligned image and a second aligned image; based on the ORB feature matching algorithm, the alignment operation consumes about 30 milliseconds.

[0074] Step 4, constructing a defect-sensitive feature space, obtaining the gray images of the first aligned image and the second aligned image, and splicing according to the channel to obtain a spliced image; in order to reduce the influence of the light factor, taking the gray image of the first aligned image as the reference, the gray image of the second aligned image is subjected to gray normalization, so that the pixel mean value of the gray image of the second aligned image after normalization is equal to the pixel mean value of the gray image of the first aligned image. Then taking the gray image of the first aligned image as the blue channel of the spliced image, the gray normalized second aligned image as the green channel of the spliced image, and the red channel of the spliced image is all set to zero. According to the spliced image obtained in this way, the sensitivity to defects can be reflected in color: if the gray printing has defects, such as printing missing, then the green channel of the spliced image presents high brightness of the background image, that is, the green channel of the spliced image presents high brightness, thereby causing the spliced image to present green; on the contrary, if the color printing has defects, such as printing missing, then the blue channel of the spliced image presents high brightness of the background image, that is, the blue channel of the spliced image presents high brightness, thereby causing the spliced image to present blue;

[0075] Step 5, training the anomaly detector using the spliced image of the defect-free sample, inputting the spliced image into the anomaly detector for anomaly detection; Padim algorithm can be used, or other unsupervised anomaly detection algorithms can be used. Feature extraction occupies the main part of Padim, and ResNet50 backbone network is used for feature extraction, which takes about 200 milliseconds on Tesla T4;

[0076] Step 6, combining the a channel feature of Lab color space on the spliced image to distinguish color or gray printing defects, realizing multi-defect collaborative detection. Specifically, the spliced image is subjected to color space conversion, converted to Lab space, and the a channel is taken to judge whether the defect position is green, if it is green, it is a gray printing result with defects, otherwise it is a color printing result with defects.

[0077] According to the above scheme, the single chamber detection time is less than 0.3 seconds.

[0078] The above are preferred embodiments of the present application, not to limit the protection scope of the present application, therefore: all equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for print defect detection based on cross-modal alignment, characterized in that, The method comprises the following steps: Step 1: acquiring a color printed image of the card as a first detection image through a visible light camera, and acquiring a gray printed image of the card as a second detection image through an infrared camera; Step 2: detecting a color printed result as a first printed target in the first detection image, and detecting a gray printed result as a second printed target in the second detection image; The first printed target is a color head-shoulder image, and the second printed target is a reduced head-shoulder image in gray scale; Step 3: aligning the first printed target and the second printed target to obtain a first aligned image and a second aligned image; Step 4: constructing a defect-sensitive feature space, acquiring a gray image of the first aligned image and the second aligned image, and splicing according to the channel to obtain a spliced image; Step 5: training an anomaly detector using a spliced image of a defect-free sample, inputting the spliced image into the anomaly detector for anomaly detection; Step 6: combining the a channel feature of the Lab color space to distinguish color or gray scale printing defects on the spliced image based on the anomaly detection result, and realizing multi-defect collaborative detection; The method for obtaining the spliced image in step 4 is: taking the gray image of the first aligned image as a reference, performing gray scale normalization on the second aligned image, so that the pixel mean value of the second aligned image is equal to the pixel mean value of the gray image of the first aligned image, to obtain the second aligned image after gray scale normalization; Taking the gray image of the first aligned image as the blue channel of the spliced image, and taking the second aligned image after gray scale normalization as the green channel of the spliced image, and setting the red channel of the spliced image to zero.

2. The method of claim 1, wherein, In step 2, a target detector is trained based on YOLOv5, and the first printed target is detected in the first detection image and the second printed target is detected in the second detection image through target detection.

3. The method of claim 1, wherein, The specific method of step 3 is: adjusting the scale of the first printed target according to the preset scale ratio of the first printed target and the second printed target, so that the adjusted first printed target and the second printed target have the same scale, and using the ORB feature point matching algorithm to register the first printed target and the second printed target, and the registered images are the first aligned image and the second aligned image respectively.

4. The method of claim 1, wherein, In step 5, the anomaly detector uses the anomaly detection algorithm Padim to realize anomaly detection.

5. The method of claim 4, wherein, The threshold value of the anomaly detector is 0.5, and the confidence map output by Padim is binarized and processed by morphological opening operation, so that the anomaly detection result can be obtained.

6. The method of claim 1, wherein, In step 6, the spliced image is converted to Lab space, the a channel is taken, and it is judged whether the defect position is green, if it is green, it is a gray scale printed result with defects, otherwise it is a color printed result with defects.

7. The method of claim 6, wherein, If a < 0, it is judged that the defect position is green.

8. Memory, characterized in that The storage adopts a printing defect detection program designed based on the printing defect detection method of any one of claims 1-7.

9. A printing defect detection apparatus based on cross-modal alignment, characterized by: The image acquisition device, the memory, the processor and the display are included, the image acquisition device includes a visible light camera and an infrared camera, the visible light camera is used for shooting and acquiring a first detection image, the infrared camera is used for shooting and acquiring a second detection image, the memory is in communication connection with the visible light camera and the infrared camera respectively, the memory stores a printing defect detection program designed based on the printing defect detection method of cross-modal alignment in any one of claims 1-7, the processor is in communication connection with the memory, runs the printing defect detection program to output a printing defect detection result, and the display is in communication connection with the processor, and the processor controls the display to display the printing defect detection result.

Images