Method, system and electronic device for detecting printing defects

Abstract

This invention relates to the field of paperboard printing technology and discloses a method, system, and electronic device for detecting printing defects. The method segments a reference drawing containing a target printed pattern using the printed surface contour to obtain a reference printing area. Responding to an image instance of the printed product on the printed surface, a perspective transformation is performed on the image instance based on the reference printing area and the printed surface contour area in the image instance to obtain an intermediate instance. A perspective transformation is then performed on the intermediate instance based on the reference edge information in the reference printing area and the current edge information in the intermediate instance to obtain a target instance. Defect areas are then extracted based on the reference printing area and the target instance. Shape matching is performed between the reference printing area and the target instance based on the connected components of the defect areas to obtain the printing defect detection result of the printed product. This achieves automated detection of printing defects, improving the accuracy and efficiency of defect detection.

Description

Technical Field

[0001] This invention relates to the field of paperboard printing technology, and in particular to a method, system and electronic device for detecting printing defects. Background Technology

[0002] Currently, various defects can occur in the printing process of corrugated cardboard, such as ink spots, foreign objects, incomplete text, missing prints, color differences, misregistration, smudges, knife marks, ink smears, scratches, glue overflow, and bubbles. Once these defects appear on product packaging, the product's visual appeal will be significantly diminished, severely impacting the brand image. To ensure product quality, customers typically agree on quality standards before printing according to the drawings. Failure to meet acceptance standards can result in economic losses ranging from tens of thousands to hundreds of thousands of yuan for the factory.

[0003] To ensure product quality, factories typically conduct meticulous manual inspections of the first printed product and perform random checks on intermediate printed products. The meticulous visual inspection of the first piece is the most crucial, requiring comparison with the drawings of every character and pattern detail, including all colors, omissions, misalignments, and printing errors. This process often takes several hours. Because the factory's order volume is not stable per unit of time, when orders are high and production capacity is tight, this process often results in significant time wastage and labor costs. Furthermore, manual proofreading of printed products is prone to misjudgment. Therefore, manual inspection of printing defects on corrugated cardboard and other printed products has low accuracy and cannot achieve the desired results. Summary of the Invention

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0005] In view of the shortcomings of the prior art described above, the present invention discloses a printing defect detection method, system and electronic device to improve the accuracy of printing defect detection.

[0006] This invention discloses a printing defect detection method, comprising: pre-segmenting a reference drawing containing a target printing pattern according to the printing surface contour of a printed product to obtain a reference printing area containing the target printing pattern; responding to an image instance of the printed product on the printing surface, extracting the contour of the printing surface in the image instance to obtain a printing surface contour area, and performing a perspective transformation on the image instance according to the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance; extracting edges from the reference printing area and the intermediate instance according to the target printing pattern to obtain reference edge information of the reference printing area and current edge information of the intermediate instance, and performing a perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain a target instance; extracting a defect area according to the reference printing area and the target instance, and performing shape matching between the reference printing area and the target instance based on the connected components of the defect area, so as to determine the printing defect detection result of the printed product according to the shape matching result.

[0007] This invention discloses a printing defect detection system, comprising: a segmentation module, used to segment a reference drawing containing a target printing pattern according to the printing surface contour of a printed product to obtain a reference printing area containing the target printing pattern; a first correction module, used to extract the contour of the printing surface in the image instance in response to an image instance of the printed product on the printing surface to obtain a printing surface contour area, and to perform perspective transformation on the image instance according to the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance; a second correction module, used to extract edges from the reference printing area and the intermediate instance according to the target printing pattern to obtain reference edge information of the reference printing area and current edge information of the intermediate instance, and to perform perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain a target instance; and a matching module, used to extract defect areas according to the reference printing area and the target instance, and to perform shape matching between the reference printing area and the target instance based on the connected components of the defect areas, so as to determine the printing defect detection result of the printed product according to the shape matching result.

[0008] The present invention discloses an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the above-described method.

[0009] The beneficial effects of this invention are:

[0010] The reference drawing containing the target printed pattern is segmented into a reference printing area by means of the printed surface contour. Responding to an image instance of the printed product on the printed surface, a perspective transformation is performed on the image instance based on the reference printing area and the printed surface contour area within the image instance to obtain an intermediate instance. Then, a perspective transformation is performed on the intermediate instance based on the reference edge information in the reference printing area and the current edge information in the intermediate instance to obtain the target instance. Defect areas are then extracted based on the reference printing area and the target instance. Shape matching is performed between the reference printing area and the target instance based on the connected components of the defect areas to obtain the printing defect detection result of the printed product. In this way, by pre-obtaining a reference printing area containing the target printed pattern, sequentially performing perspective transformations on the image instance of the printed product on the printed surface based on the printed surface contour and the edges of the printed pattern, and determining the printing defect detection result of the printed product based on the target instance and the reference printing area after two corrections, the automated detection of printing defects is achieved. Compared with manual visual inspection, this improves the accuracy and efficiency of printing defect detection. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the structure of an application environment for implementing a printing defect detection method in an embodiment of the present invention;

[0012] Figure 2 This is a flowchart illustrating a printing defect detection method according to an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram of a scalar image in an embodiment of the present invention;

[0014] Figure 4 This is a flowchart illustrating a method for obtaining a reference printing area in an embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram of a contour corner point and a rectangle vertex in an embodiment of the present invention;

[0016] Figure 6 This is a flowchart illustrating a method for obtaining an intermediate instance in an embodiment of the present invention;

[0017] Figure 7 This is a flowchart illustrating a method for obtaining a target instance in an embodiment of the present invention;

[0018] Figure 8 This is a flowchart illustrating a printing defect extraction method according to an embodiment of the present invention;

[0019] Figure 9 This is a schematic diagram of the structure of a printing defect detection system according to an embodiment of the present invention;

[0020] Figure 10This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0021] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and sub-samples in the embodiments can be combined with each other.

[0022] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0024] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0025] Unless otherwise stated, the term "multiple" means two or more.

[0026] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0027] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0028] Combination Figure 1As shown, this embodiment of the disclosure provides an application environment for implementing a printing defect detection method, including a user terminal 101 and a server 102, wherein the user terminal 101 communicates with the server 102 via a network.

[0029] User terminal 101 is used to send user commands to the server.

[0030] Server 102 is used to execute user instructions sent by the user terminal. The user instructions include at least one of the following: segmenting a reference drawing containing a target printed pattern according to the printing surface contour of the printed product to obtain a reference printing area containing the target printed pattern; in response to an image instance of the printed product on the printing surface, extracting the contour of the printing surface in the image instance to obtain a printing surface contour area, and performing a perspective transformation on the image instance according to the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance; extracting edges from the reference printing area and the intermediate instance according to the target printed pattern to obtain reference edge information of the reference printing area and current edge information of the intermediate instance, and performing a perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain a target instance; extracting a defect area according to the reference printing area and the target instance, and performing shape matching between the reference printing area and the target instance based on the connected components of the defect area to determine the printing defect detection result of the printed product according to the shape matching result.

[0031] Combination Figure 2 As shown, this disclosure provides a printing defect detection method, including:

[0032] Step S201: The reference drawing containing the target printing pattern is pre-segmented according to the printing surface outline of the printed product to obtain the reference printing area containing the target printing pattern.

[0033] Step S202: In response to the image instance of the printed product on the printed surface, the outline of the printed surface in the image instance is extracted to obtain the outline region of the printed surface, and the image instance is transformed according to the perspective transformation matrix between the reference printed area and the outline region of the printed surface to obtain the intermediate instance.

[0034] Step S203: Extract the edges from the reference printing area and the intermediate instance according to the target printing pattern to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance. Then, perform perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain the target instance.

[0035] Step S204: Extract the defect region based on the reference printing area and the target instance, and perform shape matching between the reference printing area and the target instance based on the connected components of the defect region, so as to determine the printing defect detection result of the printed product based on the shape matching result.

[0036] The printing defect detection method provided in this disclosure involves segmenting a reference drawing containing a target printed pattern using the printed surface contour to obtain a reference printing area. Responding to an image instance of the printed product on the printed surface, a perspective transformation is performed on the image instance based on the reference printing area and the printed surface contour area in the image instance to obtain an intermediate instance. Then, a perspective transformation is performed on the intermediate instance based on the reference edge information in the reference printing area and the current edge information in the intermediate instance to obtain a target instance. Defect areas are then extracted based on the reference printing area and the target instance. Shape matching is performed between the reference printing area and the target instance based on the connected components of the defect areas to obtain the printing defect detection result of the printed product. In this way, a reference printing area containing the target printed pattern is pre-obtained, and the image instance of the printed product on the printed surface is sequentially transformed based on the printed surface contour and the edges of the printed pattern. The printing defect detection result of the printed product is determined based on the target instance and the reference printing area after two corrections, thereby achieving automated detection of printing defects. Compared with manual visual inspection, this method improves the accuracy and efficiency of printing defect detection.

[0037] Optionally, the reference drawing containing the target printed pattern is segmented according to the printed surface outline of the printed product to obtain a reference printing area containing the target printed pattern. This includes: acquiring a reference drawing in vector format, the reference drawing containing the target printed pattern; calibrating the reference drawing according to the printed surface outline of the printed product to obtain region calibration marks, and converting the reference drawing with region calibration marks into a scalar image; extracting the contour from the scalar image according to the region calibration marks to obtain the printing area calibration contour, and performing dilation processing on the printing area calibration contour using a convolution kernel; extracting the calibration contour coordinates from the dilated printing area calibration contour, and then... The scalar image is segmented based on the extracted coordinates of each calibration contour, resulting in the x-coordinate and y-coordinate values ​​corresponding to each coordinate. Neighborhoods are divided for each x-coordinate value that meets a preset occurrence threshold, resulting in a first x-coordinate value set and a second x-coordinate value set. Similarly, neighborhoods are divided for each y-coordinate value that meets a preset occurrence threshold, resulting in a first y-coordinate value set and a second y-coordinate value set. The scalar image is then segmented based on the mean of the x-coordinate values ​​in the first x-coordinate value set, the mean of the x-coordinate values ​​in the second x-coordinate value set, the mean of the y-coordinate values ​​in the first y-coordinate value set, and the mean of the y-coordinate values ​​in the second y-coordinate value set, to obtain the reference printing area containing the target printing pattern.

[0038] In some embodiments, a scalar image with region calibration markings, such as Figure 3 As shown.

[0039] Optionally, the preset occurrence threshold can be determined using the following formula:

[0040] α=min(max(arrayX),max(arrayY))-β

[0041] In the formula, α is the preset occurrence threshold, arrayX is the occurrence count of each horizontal axis value, arrayY is the occurrence count of each vertical axis value, and β is the preset bias term.

[0042] In some embodiments, the preset bias is 100.

[0043] Optionally, the scalar image is segmented based on the mean of the abscissa values ​​in the first abscissa value set, the mean of the abscissa values ​​in the second abscissa value set, the mean of the ordinate values ​​in the first ordinate value set, and the mean of the ordinate values ​​in the second ordinate value set to obtain a reference printing area containing the target printing pattern. This includes: if there are multiple printing area calibration contours, sorting the printing area calibration contours according to their positions in the scalar image to obtain a contour merging order; determining each printing area calibration contour as the current contour according to the contour merging order, and determining the printing area calibration contour one position after the current contour as the subsequent contour; calculating the size difference based on the size parameters of the current contour and the subsequent contour, wherein the size parameter is a width parameter or a height parameter, and the width parameter is obtained by calculating the size difference based on the size parameters of the current contour and the subsequent contour. The height parameter is calculated by taking the mean of the x-coordinate values ​​in the first x-coordinate set and the mean of the x-coordinate values ​​in the second x-coordinate set. If the size difference is less than the preset size difference threshold, the size parameters of the subsequent contours are adjusted according to the size parameters of the current contour. If the current contour is the last printing area calibration contour, the sub-contour areas corresponding to each printing area calibration contour are obtained according to the mean of the x-coordinate values ​​in the first x-coordinate set, the mean of the x-coordinate values ​​in the second x-coordinate set, the mean of the y-coordinate values ​​in the first y-coordinate set, and the mean of the y-coordinate values ​​in the second y-coordinate set. The sub-contour areas are then merged in the order of contour merging to obtain the reference printing area containing the target printing pattern.

[0044] In some embodiments, when the number of printed area calibration contours is greater than 1, the merging order of the printed area calibration contours is determined by searching the coordinate results calculated for each printed area calibration contour in a left-to-right, top-to-bottom order. If the coordinate values ​​of the subsequent contour to be merged are similar to those of the previous contour to be merged in the horizontal or vertical direction, the size parameters of the previous contour to be merged are directly used; otherwise, the subsequent contour to be merged uses the same width or parameter as the previous contour to be merged in the width or height, and the parameters are adjusted. For the adjusted contour coordinate values, the pixel matrix information within the rectangular frame of the scalar image is obtained by sequentially segmenting from the original image, and merged from left to right in the merging order to form the standard printed area matrix. The resulting image is the standard printed area.

[0045] Combination Figure 4 As shown, this disclosure provides a method for obtaining a reference printing area, including:

[0046] Step S401: Obtain the reference drawing in vector format;

[0047] Step S402: According to the printing surface outline of the printed product, the reference drawing is calibrated to obtain the area calibration mark, and the reference drawing is converted into a scalar image;

[0048] The outline of the printed surface is marked on the base drawing in vector format using a pure black rectangular frame. If there are multiple different printed surface outlines, they are marked separately.

[0049] Among them, the scalar map conversion component is used to convert the reference drawing with area calibration marks into a scalar image with a resolution of 350 DPI;

[0050] Step S403: Extract the contour from the scalar image according to the region calibration mark to obtain the printing area calibration contour;

[0051] Among them, a pure black pixel filter is used to filter out the border line area in the scalar image, and the printing area calibration contour that meets the rectangular feature and the size meets the threshold setting is obtained from the border line area through contour extraction.

[0052] In order to avoid errors in the scalar graph conversion component, the calibration contour of the printing area is expanded by a convolution kernel;

[0053] Step S404: Extract the calibration contour coordinates from the calibration contour of the printed area after the expansion process, and obtain the horizontal and vertical coordinate values ​​corresponding to each calibration contour coordinate.

[0054] Specifically, the number of occurrences of each array element in the horizontal and vertical coordinate value arrays is determined, and array elements that meet the preset occurrence threshold are retained;

[0055] Step S405: Divide the abscissa and ordinate values ​​that meet the preset occurrence threshold into neighborhoods, and calculate x1, x2, y1, and y2 based on the neighborhood division results;

[0056] Specifically, the array elements retained in the horizontal coordinate value array and the vertical coordinate value array are divided into domains, so that the horizontal coordinate value array is divided into the first horizontal coordinate value set and the second horizontal coordinate value set, and the vertical coordinate value array is divided into the first vertical coordinate value set and the second vertical coordinate value set.

[0057] Wherein, the mean of the x-coordinate values ​​in the first x-coordinate value set is denoted as x1, the mean of the x-coordinate values ​​in the second x-coordinate value set is denoted as x2, the mean of the y-coordinate values ​​in the first y-coordinate value set is denoted as y1, and the mean of the y-coordinate values ​​in the second y-coordinate value set is denoted as y2.

[0058] Step S406: Determine whether the number of the marking outline in the printing area is one. If yes, proceed to step S407; otherwise, proceed to step S408.

[0059] Step S407: Perform image segmentation on the scalar image based on x1, x2, y1, and y2 to obtain the reference printing area containing the target printing pattern;

[0060] In this method, x1 and x2 are used as the x-coordinates of the upper left and lower right vertices of the reference printing area, respectively, and y1 and y2 are used as the y-coordinates of the upper left and lower right vertices of the reference printing area, respectively. The reference printing area is obtained by segmenting the image from the scalar image based on the upper left and lower right vertices.

[0061] Step S408: Determine whether the size difference between adjacent printed area calibration contours is less than a preset size difference threshold. If yes, proceed to step S409; otherwise, proceed to step S411.

[0062] Step S409: Adjust the dimensional parameters of the calibration contours of adjacent printing areas to be the same.

[0063] Step S410: Merge the calibration outlines of each printing area.

[0064] Step S411: Issue an error warning to the user.

[0065] Optionally, contour extraction is performed on the printed surface in the image instance to obtain the printed surface contour region, including: converting the image instance to grayscale space to obtain a first grayscale image, and performing grayscale binarization on the first grayscale image; extracting the contour of the first grayscale image after grayscale binarization according to the printed surface to obtain the grayscale contour of the printed surface; performing corner point detection on the grayscale contour of the printed surface to obtain the contour corner points of the grayscale contour of the printed surface, and calculating the minimum bounding rectangle of the grayscale contour of the printed surface to obtain the minimum bounding rectangle of the grayscale contour of the printed surface; determining the neighboring corner points corresponding to the vertices of each rectangle in the minimum bounding rectangle based on a preset distance threshold, and calculating the average corner point corresponding to each rectangle vertex to obtain the average corner point corresponding to each rectangle vertex; extracting the product contour region from the image instance based on the average corner points, wherein the average corner point is the vertex of the product contour region.

[0066] In some embodiments, the outline corner points of the grayscale profile of the printed surface and the vertices of the minimum bounding rectangle are as follows: Figure 5 As shown.

[0067] Optionally, after performing perspective transformation on the image instance based on the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance, the method further includes: pre-setting a first display interface, the first display interface including a first display layer and / or a second display layer, wherein the first display layer is used to display the intermediate instance and the second display layer is used to display the reference printing area; monitoring user commands received by the first display interface; if the user command received by the first display interface includes a click command, then exchanging the display content of the first display layer and the display content of the second display layer, wherein the click command consists of a mouse press event and a mouse release event; if the user command received by the first display interface includes a zoom command, then scaling the display content of the first display layer and the display content of the second display layer, wherein the zoom command consists of a mouse scroll wheel event.

[0068] Combination Figure 6 As shown, this disclosure provides a method for obtaining intermediate instances, including:

[0069] Step S601: Convert the image instance to grayscale space to obtain the first grayscale image;

[0070] Specifically, the image instance is mapped from the RGB (Red-Green-Blue) color space to the HSV (Hue-Saturation-Value) color space, and the V channel is extracted in the HSV color space to obtain the first grayscale image;

[0071] Step S602: Perform grayscale binarization on the first grayscale image;

[0072] Step S603: Extract the contour of the first grayscale image after grayscale binarization according to the printing surface to obtain the grayscale contour of the printing surface, and then proceed to steps S604 and S605.

[0073] Step S604: Perform corner detection on the grayscale contour of the printed surface to obtain the contour corners of the grayscale contour of the printed surface, and then proceed to step S606.

[0074] Among them, corner detection is performed on the grayscale contour of the printed surface based on Harris corner detection to obtain the contour corners of the grayscale contour of the printed surface.

[0075] Step S605: Calculate the minimum bounding rectangle of the grayscale profile of the printed surface to obtain the minimum bounding rectangle of the grayscale profile of the printed surface, and then proceed to step S606.

[0076] Step S606: Based on a preset distance threshold, determine the neighboring corner points corresponding to the vertices of each rectangle in the minimum bounding rectangle from each contour corner point;

[0077] Step S607: Calculate the average of the nearest corner points corresponding to each rectangle vertex to obtain the average corner points corresponding to each rectangle vertex.

[0078] Step S608: Extract the product outline region from the image instance based on each average corner point;

[0079] Step S609: Perform perspective transformation on the image instance based on the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance.

[0080] Step S610: Display the intermediate instance and the reference printing area through the first display interface;

[0081] The intermediate instance is displayed on the upper layer of the first display interface, and the reference printing area is displayed on the lower layer of the first display interface.

[0082] When the left mouse button is triggered, the middle instance and the baseline printing area are interchanged.

[0083] When the mouse wheel is moved, the intermediate instance and the reference printing area will be scaled according to the mouse position.

[0084] Optionally, edge extraction is performed from the reference printing area and the intermediate instance according to the target printing pattern to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, including: converting the reference printing area and the intermediate instance to grayscale space to obtain the grayscale image corresponding to the reference printing area and the grayscale image corresponding to the intermediate instance, and performing noise reduction processing on each grayscale image; and performing edge extraction on the target printing pattern in each grayscale image to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance.

[0085] Optionally, after extracting edges from the reference printing area and the intermediate instance according to the target printing pattern to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, the method further includes: pre-setting a second display interface, the second display interface including at least one of a third display layer, a fourth display layer and a fifth display layer, wherein the third display layer is used to display the current edge information, the fourth display layer is used to display the reference edge information, and the fifth display layer is used to display the preset area interval lines; dividing the intermediate instance into multiple areas to be corrected according to the area interval lines; if a movement instruction for any area to be corrected is received through the second display interface, the area to be corrected is moved according to the movement instruction.

[0086] Combination Figure 7 As shown, this disclosure provides a method for obtaining a target instance, including:

[0087] Step S701: Convert the reference printing area and the intermediate instance to grayscale space to obtain the grayscale image corresponding to the reference printing area and the grayscale image corresponding to the intermediate instance, respectively.

[0088] Step S702: Denoising is performed on each grayscale image separately;

[0089] Among them, noise reduction processing is performed on each grayscale image by mean filtering;

[0090] Step S703: Extract the edges of the target printed pattern in each grayscale image to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance.

[0091] Step S704: Perform perspective transformation on the intermediate instance based on the perspective transformation matrix between the reference edge information and the current edge information to obtain the target instance;

[0092] Step S705: Display the current edge information, reference edge information, and region interval lines through the second display interface;

[0093] Step S706: If a movement instruction for any area to be corrected is received through the second display interface, then the area to be corrected is moved according to the movement instruction.

[0094] Step S707: Output the moved target instance.

[0095] Optionally, extracting the defect region based on the reference printing area and the target instance includes: performing image processing on the reference printing area and the target instance respectively to obtain a first image corresponding to the reference printing area and a second image corresponding to the target instance, wherein the image processing includes at least a portion of grayscale space conversion, noise reduction processing, edge detection, grayscale binarization, and dilation processing; and performing XOR processing and erosion processing on the first image and the second image in sequence to obtain the defect region.

[0096] Optionally, after performing shape matching between the reference printing area and the target instance based on the connected components of the defect area to determine the printing defect detection result of the printed product according to the shape matching result, the method further includes: pre-setting a third display interface, the first display interface including a sixth display layer and / or a seventh display layer, wherein the sixth display layer is used to display the printing defect area of ​​the target instance, and the seventh display layer is used to display the reference printing area; monitoring user instructions received by the third display interface; if the user instructions received by the third display interface include a click instruction, then the display content of the sixth display layer and the display content of the seventh display layer are exchanged, wherein the click instruction consists of a mouse press event and a mouse release event; if the user instructions received by the third display interface include a zoom instruction, then the display content of the sixth display layer and the display content of the seventh display layer are scaled, wherein the zoom instruction consists of a mouse scroll wheel event.

[0097] Combination Figure 8 As shown, this disclosure provides a method for extracting printing defects, including:

[0098] Step S801: Perform grayscale conversion and noise reduction processing on the reference printing area and the target instance respectively;

[0099] Among them, noise reduction is performed based on mean filtering technology;

[0100] Step S802: Perform edge detection and grayscale binarization on the reference printing area and target instance after noise reduction.

[0101] Among them, edge detection is based on the Canny operator algorithm;

[0102] Step S803: Dilation processing is performed on the grayscale binarized reference printing area and the target instance to obtain the first image corresponding to the reference printing area and the second image corresponding to the target instance.

[0103] Among them, dilation is performed based on a 3×3 convolution kernel;

[0104] Step S804: Perform XOR processing and erosion processing on the first image and the second image in sequence to obtain the defect area;

[0105] Step S805: Extract the regions to be matched from the baseline printing region and the target instance respectively based on the connected components of the defective region;

[0106] Step S806: Perform shape matching between the regions to be matched to obtain the shape matching result;

[0107] Among them, shape matching is performed based on the geometric invariance of Hu moments;

[0108] Step S807: Determine whether the shape matching result is a successful match. If yes, proceed to step S808; otherwise, proceed to step S809.

[0109] Step S808: Determine that there are no printing defects in the printing defect detection result.

[0110] Step S809: Determine the area to be matched as the printing permission area of ​​the target instance;

[0111] Step S810: Display the printing defect area and reference printing area of ​​the target instance through the third display interface.

[0112] Optionally, the method further includes: acquiring a printed product sequence, wherein the printed product sequence includes multiple printed products arranged in printing order; determining a portion of the printed products from the printed product sequence as target products, wherein the target products include at least the first printed product in the printed product sequence; printing the target printed pattern sequentially on the printing surface of each printed product in printing order; in response to the completion of printing of any target product, acquiring the printing defect detection result of the target product and storing the printing defect detection result of the target product in a preset result database; if the printing defect detection result of the target product is that a printing defect was detected, stopping the printing of the target printed pattern; if the printing defect detection result of the target product is that no printing defect was detected, transporting the target product to a preset quality inspection storage area.

[0113] In some embodiments, during the quality inspection line initialization process, the imported reference drawing is used to extract the reference printing area at a resolution of 350 dpi.

[0114] In some embodiments, when the first printed piece in the first order of the printed product sequence comes off the production line, it is connected to the quality inspection line; when the first printed piece reaches the designated position, the quality inspection line is temporarily moved and a line scan camera is activated to take a picture of the first printed piece, obtaining an image instance of the first printed piece; an intermediate instance after one correction is obtained through a region extraction algorithm, wherein the region extraction algorithm includes extracting the contour of the printed surface in the image instance to obtain the contour region of the printed surface, and performing a perspective transformation on the image instance according to the perspective transformation matrix between the reference printed area and the contour region of the printed surface to obtain the intermediate instance; a target instance after two corrections is obtained through a secondary calibration and alignment procedure. The defect extraction algorithm is applied to the target instance and the baseline printing area to obtain a set of defective regions. The defect extraction algorithm includes extracting defective regions based on the baseline printing area and the target instance, performing shape matching between the baseline printing area and the target instance based on the connected components of the defective regions, extracting printing defective regions based on the shape matching results, and adding them to the defective region set. If the defective region set is empty, the production line is started to print the products in the printing product sequence, the first printed piece is stored in the quality inspection storage area, and the relevant information of the first printed piece during printing defect detection is stored in the database to facilitate user traceability of production problems. If the defective region set contains printing defective regions, the user is prompted to eliminate the anomaly.

[0115] In some embodiments, when a printing intermediate in the printing product sequence meets the quality inspection interval setting, it is connected to the quality inspection line; when the printing intermediate reaches the designated position, the quality inspection line is temporarily moved and a line scan camera is activated to take a picture of the printing intermediate, obtaining an image instance of the printing intermediate; a region extraction algorithm is used to obtain a first-corrected intermediate instance, wherein the region extraction algorithm includes extracting the contour of the printing surface in the image instance to obtain the printing surface contour region, and performing a perspective transformation on the image instance according to the perspective transformation matrix between the reference printing area and the printing surface contour region to obtain the intermediate instance; a second-corrected target instance is obtained through a second calibration and alignment procedure; and the target instance is then... The defect extraction algorithm is executed on the instance and the baseline printing area to obtain a set of defective regions. The defect extraction algorithm includes extracting defective regions based on the baseline printing area and the target instance, performing shape matching between the baseline printing area and the target instance based on the connected components of the defective regions, extracting printing defective regions based on the shape matching results, and adding them to the defective region set. If the defective region set is empty, the printing intermediate is transported back to the production line, and other printing products in the printing product sequence are printed. If the defective region set contains printing defective regions, the production line is stopped, the printing intermediate is stored in the quality inspection storage area, and the relevant information of the printing intermediate during printing defect detection is stored in the database to facilitate workers to check for problems.

[0116] Combination Figure 9 As shown, this embodiment of the present disclosure provides a printing defect detection system, including a slitting module 901, a first correction module 902, a second correction module 903, and a matching module 904.

[0117] The segmentation module 901 is used to segment the reference drawing containing the target printing pattern according to the printing surface outline of the printed product, so as to obtain the reference printing area containing the target printing pattern.

[0118] The first correction module 902 is used to respond to an image instance of a printed product on a printed surface, extract the contour of the printed surface in the image instance to obtain the contour region of the printed surface, and perform perspective transformation on the image instance according to the perspective transformation matrix between the reference printed area and the contour region of the printed surface to obtain an intermediate instance.

[0119] The second correction module 903 is used to extract edges from the reference printing area and the intermediate instance according to the target printing pattern, obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, and perform perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain the target instance.

[0120] The matching module 904 is used to extract the defect region based on the reference printing area and the target instance, and to perform shape matching between the reference printing area and the target instance based on the connected components of the defect region, so as to determine the printing defect detection result of the printed product based on the shape matching result.

[0121] The printing defect detection system provided in this disclosure performs image segmentation on a reference drawing containing a target printed pattern using the printed surface contour to obtain a reference printing area. Responding to an image instance of the printed product on the printed surface, a perspective transformation is performed on the image instance based on the reference printing area and the printed surface contour area in the image instance to obtain an intermediate instance. A perspective transformation is then performed on the intermediate instance based on the reference edge information in the reference printing area and the current edge information in the intermediate instance to obtain a target instance. Defect areas are then extracted based on the reference printing area and the target instance. Shape matching is performed between the reference printing area and the target instance based on the connected components of the defect areas to obtain the printing defect detection result of the printed product. In this way, a reference printing area containing the target printed pattern is pre-obtained, and the image instance of the printed product on the printed surface is sequentially transformed based on the printed surface contour and the edges of the printed pattern. The printing defect detection result of the printed product is determined based on the target instance after two corrections and the reference printing area, thereby achieving automated detection of printing defects. Compared with manual visual inspection, this improves the accuracy and efficiency of printing defect detection.

[0122] This disclosure also provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the above-described method.

[0123] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0124] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from Storage Unit 1008 into Random Access Memory (RAM) 1003. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0125] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. Drive 1100 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1100 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0126] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0127] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0128] The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic device performs the various steps of the above method.

[0129] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used herein means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated subsamples, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other subsamples, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0131] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some sub-samples may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for detecting printing defects, characterized in that, include: The reference drawing containing the target printing pattern is pre-segmented according to the printing surface outline of the printed product to obtain the reference printing area containing the target printing pattern; In response to an image instance of the printed product on the printed surface, the outline of the printed surface in the image instance is extracted to obtain a printed surface outline region, and the image instance is subjected to perspective transformation according to the perspective transformation matrix between the reference printed area and the printed surface outline region to obtain an intermediate instance. Edges are extracted from the reference printing area and the intermediate instance according to the target printing pattern to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, respectively. The intermediate instance is then subjected to perspective transformation based on the perspective transformation matrix between the reference edge information and the current edge information to obtain the target instance. Defect regions are extracted based on the reference printing area and the target instance, and shape matching is performed between the reference printing area and the target instance based on the connected components of the defect regions, so as to determine the printing defect detection result of the printed product based on the shape matching result.

2. The method according to claim 1, characterized in that, The reference drawing containing the target printed pattern is segmented according to the printed surface outline of the printed product to obtain a reference printing area containing the target printed pattern, including: Obtain a reference drawing in vector graphic format, the reference drawing containing the target printing pattern; The reference drawing is calibrated according to the outline of the printed surface of the printed product to obtain the region calibration mark, and the reference drawing with the region calibration mark is converted into a scalar image; Contours are extracted from the scalar image according to the region calibration marks to obtain the printing area calibration contours, and the printing area calibration contours are dilated by a convolution kernel; The calibration contour coordinates are extracted from the calibration contour of the printed area after the expansion process, and the extracted calibration contour coordinates are split according to the coordinate dimensions to obtain the horizontal and vertical coordinate values ​​corresponding to each calibration contour coordinate. The neighborhood of each horizontal coordinate value that meets the preset occurrence threshold is divided to obtain a first horizontal coordinate value set and a second horizontal coordinate value set. Similarly, the neighborhood of each vertical coordinate value that meets the preset occurrence threshold is divided to obtain a first vertical coordinate value set and a second vertical coordinate value set. The scalar image is segmented based on the mean of the x-coordinate values ​​in the first x-coordinate value set, the mean of the x-coordinate values ​​in the second x-coordinate value set, the mean of the y-coordinate values ​​in the first y-coordinate value set, and the mean of the y-coordinate values ​​in the second y-coordinate value set to obtain a reference printing area containing the target printing pattern.

3. The method according to claim 2, characterized in that, The scalar image is segmented based on the mean of the x-coordinate values ​​in the first x-coordinate value set, the mean of the x-coordinate values ​​in the second x-coordinate value set, the mean of the y-coordinate values ​​in the first y-coordinate value set, and the mean of the y-coordinate values ​​in the second y-coordinate value set to obtain a reference printing area containing the target printing pattern, including: If there are multiple printing area calibration contours, then the printing area calibration contours are sorted according to their positions in the scalar image to obtain the contour merging order; According to the outline merging order, each of the printing area marking outlines is sequentially determined as the current outline, and the printing area marking outline that follows the current outline is determined as the subsequent outline. The size difference is calculated based on the size parameters of the current contour and the size parameters of the subsequent contour. The size parameters are either width parameters or height parameters. The width parameter is calculated by the mean of the x-coordinate values ​​in the first x-coordinate value set and the mean of the x-coordinate values ​​in the second x-coordinate value set. The height parameter is calculated by the mean of the y-coordinate values ​​in the first y-coordinate value set and the mean of the y-coordinate values ​​in the second y-coordinate value set. If the size difference is less than a preset size difference threshold, then the size parameters of the subsequent contours are adjusted according to the size parameters of the current contour. If the current contour is the last printing area calibration contour, then the sub-contour regions corresponding to each printing area calibration contour are obtained according to the mean of the contour abscissa values ​​of the first abscissa value set, the mean of the contour abscissa values ​​of the second abscissa value set, the mean of the contour ordinate values ​​of the first ordinate value set, and the mean of the contour ordinate values ​​of the second ordinate value set. Then, the sub-contour regions are merged in sequence according to the contour merging order to obtain the reference printing area containing the target printing pattern.

4. The method according to claim 1, characterized in that, The printed surface in the image instance is contour extracted to obtain the printed surface contour region, including: The image instance is converted to grayscale space to obtain a first grayscale image, and the first grayscale image is then subjected to grayscale binarization. The first grayscale image after grayscale binarization is used to extract the contour of the printed surface to obtain the grayscale contour of the printed surface. Corner point detection is performed on the grayscale profile of the printed surface to obtain the profile corner points of the grayscale profile of the printed surface, and the minimum bounding rectangle is calculated on the grayscale profile of the printed surface to obtain the minimum bounding rectangle of the grayscale profile of the printed surface. Based on a preset distance threshold, the neighboring corner points corresponding to each rectangle vertex in the minimum bounding rectangle are determined from each of the contour corner points, and the average value of the neighboring corner points corresponding to each rectangle vertex is calculated to obtain the average corner point corresponding to each rectangle vertex. Product outline regions are extracted from the image instance based on the average corner points, wherein the average corner points are the vertices of the product outline regions.

5. The method according to claim 1, characterized in that, After performing a perspective transformation on the image instance based on the perspective transformation matrix between the reference printing area and the printing surface contour area to obtain an intermediate instance, the method further includes: A first display interface is pre-set, the first display interface including a first display layer and / or a second display layer, wherein the first display layer is used to display the intermediate instance, and the second display layer is used to display the reference printing area; Monitor user commands received by the first display interface; If the user instruction received by the first display interface includes a click instruction, then the display content of the first display layer and the display content of the second display layer are exchanged. The click instruction consists of a mouse press event and a mouse release event. If the user command received by the first display interface includes a zoom command, then the display content of the first display layer and the display content of the second display layer will be scaled. The zoom command consists of a mouse wheel event.

6. The method according to claim 1, characterized in that, After extracting edges from the reference printing area and the intermediate instance according to the target printing pattern to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, the method further includes: A second display interface is preset, which includes at least one of a third display layer, a fourth display layer, and a fifth display layer. The third display layer is used to display the current edge information, the fourth display layer is used to display the reference edge information, and the fifth display layer is used to display preset area interval lines. The intermediate instance is divided into multiple regions to be corrected according to the region interval lines; If a movement instruction for any area to be corrected is received through the second display interface, the area to be corrected is moved according to the movement instruction.

7. The method according to claim 1, characterized in that, Defect regions are extracted based on the baseline printing area and the target instance, including: Image processing is performed on the reference printing area and the target instance respectively to obtain a first image corresponding to the reference printing area and a second image corresponding to the target instance. The image processing includes at least a portion of grayscale space conversion, noise reduction, edge detection, grayscale binarization, and dilation. The first image and the second image are XORed and eroded sequentially to obtain the defect area.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Obtain a sequence of printed products, wherein the sequence of printed products includes multiple printed products arranged in printing order; A portion of the printed products in the printed product sequence are selected as target products, wherein the target products include at least the first printed product in the printed product sequence; The target printed pattern is printed sequentially on the printing surface of each of the printed products in the described printing order; In response to the completion of printing of any target product, the printing defect detection result of the target product is obtained and stored in a preset result database; If the printing defect detection result of the target product is that a printing defect is detected, then the printing of the target printed pattern shall be stopped; If the printing defect detection result of the target product is that no printing defect is detected, the target product will be transported to the preset quality inspection storage area.

9. A printing defect detection system, characterized in that, include: The segmentation module is used to segment the reference drawing containing the target printing pattern according to the printing surface outline of the printed product to obtain the reference printing area containing the target printing pattern. The first correction module is used to extract the contour of the printed surface in the image instance in response to the image instance of the printed product on the printed surface, to obtain the printed surface contour region, and to perform perspective transformation on the image instance according to the perspective transformation matrix between the reference printed area and the printed surface contour region to obtain an intermediate instance. The second correction module is used to extract edges from the reference printing area and the intermediate instance according to the target printing pattern, to obtain the reference edge information of the reference printing area and the current edge information of the intermediate instance, and to perform perspective transformation on the intermediate instance according to the perspective transformation matrix between the reference edge information and the current edge information to obtain the target instance. The matching module is used to extract defect regions based on the reference printing area and the target instance, and to perform shape matching between the reference printing area and the target instance based on the connected components of the defect regions, so as to determine the printing defect detection result of the printed product based on the shape matching result.

10. An electronic device, characterized in that, include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 8.

Images