Printing quality visual detection method for reflective product, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of image detection, and particularly discloses a printing quality visual detection method for a reflective product, electronic equipment and a storage medium, and the method comprises the steps: S1, collecting product image data with a reflective region, obtaining an ideal segmentation effect image through processing, and making a data set; s2, training and testing a semantic segmentation network model k-net; s3, collecting an image of a defect-free product, preprocessing the image, inputting the image into the trained semantic segmentation network model k-net, and processing the image to make a template image set; s4, preprocessing a to-be-detected image, and inputting the preprocessed to-be-detected image into the trained semantic segmentation network model k-net to obtain a segmentation result; s5, carrying out rough matching on the segmentation result and a template, and intercepting a region of interest in the segmentation result; and S6, in the region of interest of the segmentation result, performing fine matching on each printing body, checking the printing quality of each printing body, and completing the printing quality detection of the product.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular relates to a printing quality visual detection method for reflective products, an electronic device and a storage medium. Background Art

[0002] Visual inspection technology has become one of the most widely used technologies in the field of industrial production and manufacturing due to its many advantages such as accurate perception, high production efficiency, stable inspection results, and can be used in dangerous scenarios and diversified inspection objects. For modern enterprises, the application of machine vision inspection technology is the only way for enterprises to transform and upgrade to intelligent manufacturing. In the manufacturing and assembly processes, it can greatly improve production efficiency, improve product quality, and enhance the market competitiveness of enterprises.

[0003] However, most of the visual inspection systems on the market are based on traditional image processing methods, and generally have problems such as poor adaptability to products and high false alarm rate. In addition, in industrial environments, under the influence of strong reflective interference factors such as light and product material, reflective areas caused by strong reflections usually appear on the product surface, which affects the detection of printing quality. Traditional visual inspection algorithms are difficult to effectively process reflective areas, resulting in the following problems in printing quality inspection: First, in the collected image, part of the area to be inspected will overlap with the reflective area, making it difficult to distinguish the printed body in the area to be inspected from the reflective area, which can easily lead to missed detection of printing defects or false detection of qualified products; second, when the size of printing flaws and defects is relatively small relative to the collected image, traditional algorithms such as template matching, threshold segmentation and other methods usually cannot achieve high-precision positioning, making it difficult to detect printing flaws and defects, resulting in defects in the products after visual inspection, affecting the quality of the products after inspection. Summary of the invention

[0004] Based on the above problems, the present invention provides a method, electronic device and storage medium for visual inspection of printing quality of reflective products.

[0005] Based on the above purpose, the present invention is achieved through the following technical solutions:

[0006] A first aspect of the present invention provides a method for visually inspecting printing quality of a reflective product, comprising the following steps:

[0007] S1. Collect image data of products with reflective areas, obtain ideal segmentation effect images after processing, and create a data set.

[0008] S2. Train and test the semantic segmentation network model k-net.

[0009] S3. Collect images of defect-free products, input them into the trained semantic segmentation network model k-net after preprocessing, and produce a template image set after processing.

[0010] S4. The image to be detected is preprocessed and input into the trained semantic segmentation network model k-net to obtain the segmentation result.

[0011] S5. Roughly match the segmentation result with the template and extract the region of interest in the segmentation result.

[0012] S6. In the region of interest of the segmentation result, each printed body is precisely matched, and the printing quality of each printed body is checked to complete the printing quality inspection of the product.

[0013] According to the above-mentioned visual inspection method for printing quality of reflective products, preferably, in step S1, the data set includes a data set of original images of products with reflective areas and a data set of corresponding ideal segmentation effect images, and the data set is divided into a training data set and a test data set in a ratio of 8:2; wherein, the data set of original images of products is obtained by shooting, storing, and preprocessing with an industrial camera in a stable environment, and the preprocessing content of the data set of original images of products includes image scaling and contrast enhancement.

[0014] According to the above-mentioned printing quality visual inspection method for reflective products, preferably, the step of obtaining an ideal segmentation effect image through processing includes:

[0015] S101, after the original product image is binarized, a binary image in which the printed body and the reflective area coexist is obtained.

[0016] S102, converting the binary image in which the printed body and the reflective area coexist into a file in Json format.

[0017] S103. Use the image annotation software Labelme to display the Json file in the form of labels, and manually correct the printed body label in the image annotation software Labelme to ensure that only the printed body is labeled and the reflective area is not labeled, label all the printed bodies with the same label, and label the rest including the reflective area as the background label.

[0018] S104, converting the corrected Json file into a binary image to obtain a corresponding ideal segmentation effect image.

[0019] According to the above-mentioned visual inspection method for print quality of reflective products, preferably, the specific steps of step S2 include: using the original image of the product and the ideal segmentation effect image in the training data set to train the semantic segmentation network model k-net, guiding the network to learn to distinguish the features of the printed body and the reflective area, and testing the model performance of the semantic segmentation network model k-net after the training is completed to ensure that the printed body can be completely segmented under the interference of the reflective area.

[0020] According to the above-mentioned printing quality visual inspection method for reflective products, preferably, the specific steps of step S3 include:

[0021] S301, collecting defect-free product images, and inputting them into the semantic segmentation network model k-net that has been trained in step S2 after the same preprocessing as step S1, to obtain the segmentation mask of the printed body in the product.

[0022] S302: extracting contours in the segmentation mask, calculating contour areas, and setting thresholds to remove small noise points in the segmentation mask to improve template quality.

[0023] S303: Since the analysis of the entire image will bring about a large computational redundancy, the mask image is cropped by calculating the minimum bounding rectangle containing all segmentation masks to form a mask image of the region of interest, which is used as a template image for subsequent coarse matching.

[0024] S304, performing connected domain analysis in the region of interest image, intercepting and saving each connected domain as a separate template for subsequent precise matching of each printed body, and completing the production of a product template image set.

[0025] According to the above-mentioned printing quality visual inspection method for reflective products, preferably, the specific steps of step S5 include:

[0026] S501, rotating the mask template image of the region of interest obtained in step S303 according to a series of angles, performing NCC matching on the template images at different angles and the segmentation results of the product to be inspected in step S4, respectively, to obtain the best matching angle and matching area;

[0027] NCC is:

[0028]

[0029] Where T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the average value of the template image; It is the average value of the corresponding window at position (u, v) of the target image.

[0030] S502: Using the best matching angle obtained in step S501, the segmentation mask of the image to be detected is rotated to adapt to the angle change of the product.

[0031] S503 , using the matching region coordinates obtained in step S501 , intercepting the region of interest of the segmentation mask of the image to be detected.

[0032] According to the above-mentioned printing quality visual inspection method for reflective products, preferably, the specific steps of step S6 include:

[0033] S601, using the individual template of each printed body obtained in step S304 to perform NCC template matching in the region of interest obtained in step S5, setting a threshold according to actual detection requirements, screening out the printing quality defect area of ​​the printed body according to the threshold, and judging whether there is missing or missed printing.

[0034] S602, after smearing the matched area in step S601, perform connected domain analysis again, calculate the contour area, and set a threshold to remove small noise points in the segmentation mask. After filtering the noise points, determine whether there are multiple printed bodies, and determine whether there are multiple printed bodies.

[0035] According to the above-mentioned visual inspection method for printing quality of reflective products, preferably, the specific steps of step S4 include: using the industrial camera in step S1 to obtain the image to be inspected, and inputting it into the trained semantic segmentation network model k-net after the same preprocessing as step S1 to obtain the segmentation result of the product to be inspected.

[0036] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any step of the method for visually inspecting printing quality of reflective products as described in the first aspect is implemented.

[0037] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a computer processor, any step of the method for visual inspection of printing quality of reflective products as described in the first aspect is implemented.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention uses a trained semantic segmentation network model k-net to perform semantic segmentation on the printed body of the product covered by the reflective area, which can effectively remove the reflective area on the product surface, reduce the influence of strong reflective interference on visual detection, and improve the visual detection effect of product printing under strong reflective interference; at the same time, by rotating and matching the mask template image of the product's area of ​​interest, the angle difference between the template image and the image to be detected can be corrected, thereby solving the problem of false detection caused by the change of the product angle; and then by matching with a separate template of each printed body, the printing quality is detected, and the printing quality defect area of ​​the printed body can be screened out, so as to realize the judgment of the lack of printing, missed printing, etc. of the product printed body. Therefore, the method of the present invention can significantly improve the anti-reflective interference ability of visual detection, and can significantly improve the accuracy and efficiency of product detection, and ensure the quality of the product after detection.

[0040] 2. The method provided by the present invention can accurately complete the task of de-reflecting printed body segmentation for the printed body covered by the reflective area of ​​the product, thereby effectively preventing the phenomenon of interference with visual inspection due to the reflective area covering the printed area, and can improve the accuracy and efficiency of visual inspection of product printing quality, effectively improve product printing quality, and avoid problems such as printing flaws and defects in products. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a schematic diagram of the process of Example 1 of the present invention;

[0042] Figure 2 This is a specific implementation flow chart of Example 1 of the present invention;

[0043] Figure 3 The original product image and the corresponding ideal segmentation effect image in Example 1 of the present invention, wherein the left side is the original product image and the right side is the ideal segmentation effect image;

[0044] Figure 4 is the mask template image of the region of interest in Embodiment 1 of the present invention;

[0045] Figure 5 It is a partial single printed body mask template image in Embodiment 1 of the present invention;

[0046] Figure 6 is the segmentation mask image of the product to be detected in Example 1 of the present invention;

[0047] Figure 7 This is the printing quality test result of the product to be tested in Example 1 of the present invention; wherein, the green frame represents the printing quality qualified area, the red frame represents the missed printing or missing printing area, and the blue frame represents the over-printed area. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below through specific examples, but the scope of the present invention is not limited thereto.

[0049] Example 1

[0050] A visual inspection method for printing quality of reflective products, the process is as follows Figure 1 The specific implementation process is as shown in Figure 2 As shown, it mainly includes two stages: template making and quality inspection; including the following steps:

[0051] S1. Collect image data of products with reflective areas, obtain ideal segmentation effect images after processing, and create a data set.

[0052] The data set includes a data set of original images of products with reflective areas and a data set of corresponding ideal segmentation effect images. The data sets of original images of products and corresponding ideal segmentation effect images in the data set are as follows: Figure 3 As shown, the data set includes 450 pairs, of which 360 pairs of image data are training sets and 90 pairs of image data are test sets; among them, the product original image data set is shot and stored by an industrial camera in a stable environment, and is obtained after preprocessing. The preprocessing content of the product original image data set includes image scaling and contrast enhancement.

[0053] The step of obtaining an ideal segmentation effect image through processing comprises:

[0054] S101, after the original product image is binarized, a binary image in which the printed body and the reflective area coexist is obtained.

[0055] S102, converting the binary image in which the printed body and the reflective area coexist into a file in Json format.

[0056] S103. Use the image annotation software Labelme to display the Json file in the form of labels, and manually correct the printed body label in the image annotation software Labelme to ensure that only the printed body is labeled and the reflective area is not labeled, label all the printed bodies with the same label, and label the rest including the reflective area as the background label.

[0057] S104, converting the corrected Json file into a binary image to obtain a corresponding ideal segmentation effect image.

[0058] S2. Train and test the semantic segmentation network model k-net.

[0059] The original product images and ideal segmentation effect images in the training data set are used to train the semantic segmentation network model k-net, guiding the network to learn to distinguish the features of printed matter and reflective areas. After the training, the model performance of the semantic segmentation network model k-net is tested to ensure that the printed matter can be completely segmented under the interference of the reflective area.

[0060] S3. Collect images of defect-free products, input them into the trained semantic segmentation network model k-net after preprocessing, and make a template image set after processing:

[0061] S301, collecting defect-free product images, and inputting them into the semantic segmentation network model k-net that has been trained in step S2 after the same preprocessing as step S1, to obtain the segmentation mask of the printed body in the product.

[0062] S302, extracting the contours in the segmentation mask, calculating the contour area, and removing small noise points with an area less than 20 units to improve the template quality.

[0063] S303: Since the analysis of the entire image will bring about a large amount of computational redundancy, the mask image is cropped by calculating the minimum bounding rectangle containing all segmentation masks to form the following: Figure 4 The region of interest mask template image shown is used as the template image for subsequent coarse matching.

[0064] S304, performing connected domain analysis in the region of interest image, intercepting and saving each connected domain, such as Figure 5 The partial individual printed body mask template image shown is a schematic diagram of a partially extracted connected domain, which serves as an individual template for subsequent precise matching of each printed body.

[0065] S4. The image to be detected is preprocessed and input into the trained semantic segmentation network model k-net to obtain the segmentation result.

[0066] The image to be inspected is obtained by an industrial camera, and after the same preprocessing as step S1, it is input into the trained semantic segmentation network model k-net to obtain the segmentation result of the product to be inspected. The segmentation result is as follows: Figure 6 shown.

[0067] S5. Roughly match the segmentation result with the template and extract the region of interest in the segmentation result:

[0068] S501, rotating the mask template image of the region of interest obtained in step S303 according to a series of angles, performing NCC matching on the template images at different angles and the segmentation results of the product to be inspected in step S4, respectively, to obtain the best matching angle and matching area;

[0069] NCC is:

[0070]

[0071] Where T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the average value of the template image; It is the average value of the corresponding window at position (u, v) of the target image.

[0072] S502: Using the best matching angle obtained in step S501, the segmentation mask of the image to be detected is rotated to adapt to the angle change of the product.

[0073] S503 , using the matching region coordinates obtained in step S501 , intercepting the region of interest of the segmentation mask of the image to be detected.

[0074] S6. In the region of interest of the segmentation result, each printed body is precisely matched, and the printing quality of each printed body is checked to complete the printing quality inspection of the product. The product printing quality inspection result is as follows: Figure 7 As shown, the green frame represents the area with qualified printing quality, the red frame represents the missed printing or missing printing area, and the blue frame represents the over-printed area.

[0075] S601, using the individual template of each printed body obtained in step S304 to perform NCC template matching in the region of interest obtained in step S5, setting a threshold according to actual detection requirements, screening out the printing quality defect area of ​​the printed body according to the threshold, and judging whether there is missing or missed printing.

[0076] S602, after smearing the matched area in step S601, perform connected domain analysis again, calculate the contour area, and set a threshold to remove small noise points in the segmentation mask. After filtering the noise points, determine whether there is any overprinted printed body, so as to realize the judgment of whether there is any overprinting of the printed body.

[0077] It can be seen from the above results that the present application can accurately complete the task of de-reflecting printed body segmentation for the printed body covered by the reflective area of ​​the product, thereby effectively preventing the phenomenon of interference with visual inspection due to the reflective area covering the printed area, facilitating the effective identification of the printed body in the reflective area, and being able to solve the problem of false detection caused by changes in the product angle, thereby improving the accuracy and efficiency of visual inspection of product printing quality, effectively improving product printing quality, and avoiding problems such as printing flaws and defects in products.

[0078] Example 2

[0079] An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, any step of a method for visually inspecting printing quality of a reflective product as described in Example 1 is implemented.

[0080] Further, the process of the method for visual inspection of print quality of reflective products described in Embodiment 1 can be implemented as a computer software program. For example, this embodiment includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, the above functions defined in the method of the present application are executed.

[0081] Example 3

[0082] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step in a method for visually inspecting printing quality of a reflective product as described in Example 1.

[0083] The computer-readable medium described in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0084] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] In summary, the present invention effectively overcomes the deficiencies in the prior art and has a high industrial utilization value. The above embodiments serve to illustrate the substantive content of the present invention, but do not limit the protection scope of the present invention. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and protection scope of the technical solution of the present invention.

[0086] The above embodiments are specific implementation methods of the present invention, but the implementation methods of the present invention are not limited to the above embodiments. Any other combination, change, modification, substitution and simplification that does not exceed the design concept of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for visual inspection of printing quality of reflective products, characterized in that: The following steps are involved: S1. Collect image data of products with reflective areas, obtain ideal segmentation effect images after processing, and create a data set; S2, train and test the semantic segmentation network model k-net; S3, collect images of defect-free products, input them into the trained semantic segmentation network model k-net after preprocessing, and make a template image set after processing; S4, input the pre-processed image to be detected into the trained semantic segmentation network model k-net to obtain the segmentation result; S5, roughly matching the segmentation result with the template, and intercepting the region of interest in the segmentation result; S6. In the region of interest of the segmentation result, each printed body is precisely matched, and the printing quality of each printed body is checked to complete the printing quality inspection of the product.

2. The method for visual inspection of printing quality of reflective products according to claim 1, characterized in that: In step S1, the data set includes a data set of original images of products with reflective areas and a data set of corresponding ideal segmentation effect images, and the data set is divided into a training data set and a test data set in a ratio of 8:2; wherein, the data set of original images of products is obtained by shooting, storing, and preprocessing with an industrial camera in a stable environment, and the preprocessing content of the data set of original images of products includes image scaling and contrast enhancement.

3. The method for visual inspection of printing quality of reflective products according to claim 2, characterized in that: The step of obtaining an ideal segmentation effect image through processing comprises: S101, after the original image of the product is binarized, a binary image in which the printed body and the reflective area coexist is obtained; S102, converting the binary image in which the printed body and the reflective area coexist into a file in Json format; S103, using image annotation software to display the Json file in the form of labels, and manually correcting the printed body labels in the image annotation software to ensure that only the printed body is labeled and the reflective area is not labeled, all the printed bodies are labeled with the same label, and the rest of the parts including the reflective area are labeled as background labels; S104, converting the corrected Json file into a binary image to obtain a corresponding ideal segmentation effect image.

4. The method for visual inspection of printing quality of reflective products according to claim 2, characterized in that: The specific steps of step S2 include: using the original product images and ideal segmentation effect images in the training data set to train the semantic segmentation network model k-net, guiding the network to learn to distinguish the features of printed matter and reflective areas, and testing the model performance of the semantic segmentation network model k-net after the training is completed to ensure that the printed matter can be completely segmented under the interference of the reflective area.

5. The method for visual inspection of printing quality of reflective products according to claim 2, characterized in that: The specific steps of step S3 include: S301, collecting defect-free product images, and inputting them into the semantic segmentation network model k-net that has been trained in step S2 after the same preprocessing as step S1, to obtain the segmentation mask of the printed body in the product; S302, extracting the contour in the segmentation mask, calculating the contour area, and setting a threshold to remove small noise points in the segmentation mask to improve the template quality; S303, cropping the mask image by calculating the minimum bounding rectangle containing all segmentation masks to form a mask image of the region of interest as a template image for subsequent coarse matching; S304, performing connected domain analysis in the region of interest image, intercepting and saving each connected domain as a separate template for subsequent precise matching of each printed body, and completing the production of a product template image set.

6. The method for visual inspection of printing quality of reflective products according to claim 5, characterized in that: The specific steps of step S5 include: S501, rotating the mask template image of the region of interest obtained in step S303 according to a series of angles, performing NCC matching on the template images at different angles and the segmentation results of the product to be inspected in step S4, respectively, to obtain the best matching angle and matching area; NCC is: Where T(x,y) is the pixel value of the template image; I(x+u,y+v) is the pixel value of the target image at position (u,v); is the average value of the template image; is the average value of the corresponding window at position (u, v) of the target image; S502, using the best matching angle obtained in step S501 to rotate the segmentation mask of the image to be detected; S503 , using the matching region coordinates obtained in step S501 , intercepting the region of interest of the segmentation mask of the image to be detected.

7. The method for visual inspection of printing quality of reflective products according to claim 5, characterized in that: The specific steps of step S6 include: S601, using the individual template of each printed body obtained in step S304 to perform NCC template matching in the area of ​​interest obtained in step S5, setting a threshold according to actual detection requirements, screening out the printing quality defect area of ​​the printed body according to the threshold, and judging whether there is missing or missed printing; S602, after smearing the matched area in step S601, perform connected domain analysis again, calculate the contour area, and set a threshold to remove small noise points in the segmentation mask. After filtering the noise points, determine whether there are multiple printed bodies, and determine whether there are multiple printed bodies.

8. The method for visual inspection of printing quality of reflective products according to claim 2, characterized in that: The specific steps of step S4 include: using an industrial camera to obtain the image to be detected, and inputting it into a trained semantic segmentation network model k-net after the same preprocessing as step S1 to obtain the segmentation result of the product to be detected.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, any step of the method for visually inspecting printing quality of reflective products according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer processor, any step of the method for visually inspecting printing quality of reflective products according to any one of claims 1 to 8 is implemented.

Images