A method for detecting defects in deformed paperboard

Abstract

The present invention discloses a method for detecting defects of distorted cardboard, comprising the following steps: S1, collecting template images: photographing and collecting defect-free product images as template images; S2, collecting images to be tested: photographing and collecting images to be tested; S3, detecting large pattern areas of images to be tested: after selecting a large pattern area in the image to be tested collected in step S2, matching the template image in step S1 to detect the large pattern area; S4, detecting small pattern areas of images to be tested: after selecting a small pattern area in the image to be tested collected in step S2, matching the template image in step S1 to detect the small pattern area; S5, outputting a determination result. The present invention adopts independent detection methods for large pattern areas and small pattern areas for detection and determination, effectively improving the accuracy of defect detection and improving detection efficiency.

Description

Technical Field

[0001] The invention relates to the field of algorithms, and in particular to a method for detecting defects in distorted paperboard. Background Art

[0002] With the tremendous development of modern logistics packaging, the output of cardboard printing products has increased dramatically, and the quality inspection of printed products has gradually transitioned from manual inspection to machine inspection.

[0003] Compared with general thin paper printing, cardboard printing has its own particularity. Since cardboard is relatively thick, the difference in moisture content of raw materials and heating during the production process will cause inevitable deformation of the finished cardboard product, resulting in local distortion of the cardboard photographed image; conventional defect detection algorithms based on template matching are very sensitive to local errors in distorted images, which will lead to false detections, thus seriously affecting the accuracy of machine quality inspection. In view of this, it is necessary to design a defect detection method that can effectively improve detection accuracy. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for detecting distorted cardboard defects by using independent detection methods for large pattern areas and small pattern areas in response to the deficiencies of the above-mentioned prior art, thereby effectively improving the accuracy of defect detection and improving detection efficiency.

[0005] The technical solution adopted by the present invention is as follows: A method for detecting defects in distorted paperboard comprises the following steps:

[0006] S1. Acquiring a template image: capturing a defect-free product image as a template image;

[0007] S2, collecting the image to be tested: photographing and collecting the image to be tested;

[0008] S3, large pattern area detection of the image to be tested: after the large pattern area is selected in the image to be tested acquired in step S2, the large pattern area is detected by matching the template image in step S1;

[0009] S4, small pattern area detection of the image to be tested: after selecting the small pattern area in the image to be tested collected in step S2, the template image in step S1 is matched to perform small pattern area detection;

[0010] S5, outputting the determination result: when the detection results outputted in step S3 and step S4 are both defect-free, it is determined that the image to be tested is defect-free; when the detection results outputted in step S3 and / or step S4 are defective, it is determined that the image to be tested is defective;

[0011] In the process of detecting the large pattern area in step S3, first, a template matching area M1 is selected on the template image T, and the feature values ​​of the pixels in the template matching area M1 are calculated to form an edge shape feature; in addition, a mask image Tm is obtained after marking the small pattern area in the template image T; secondly, a corresponding test matching area M2 is selected in the acquired image A, and the feature values ​​of the pixels in the test matching area M2 are calculated to form an edge shape feature; then, by comparing the feature points of the image A to be tested with those of the template image T, the posture adjustment of the image A to be tested is completed to obtain an adjustment graph Ar; after the template image T is range-binarized to form an upper limit graph Tup and a lower limit graph Tlow respectively, the adjustment graph Ar is range-binarized with the upper limit graph Tup and the lower limit graph Tlow to form a mask image Amask; the mask image Amask and the mask image Tm are image-calculated to obtain a difference graph E; defect detection is performed in the difference graph E through the area threshold C to determine the defects of the large pattern part of the image A to be tested, and the determination result is output.

[0012] Preferably, when calculating the characteristic values ​​of the pixels in the template matching area M1 or the test matching area M2 in step S3, a convolution calculation method is adopted, that is, the pixel values ​​of the pixels in the template matching area M1 or the test matching area M2 are weighted and summed. The calculation process is to multiply each pixel value by each element of the convolution template respectively, and sum the convolution results. The calculated sum is the result of the convolution operation. The characteristic value calculation formula is as follows:

[0013]

[0014] Among them, g(i,j) is the calculated eigenvalue corresponding to the pixel point, f is the image matrix, (i,j) is the coordinate of f, h is the convolution template, and (k,l) is the coordinate of h.

[0015] Preferably, after the characteristic value is obtained by calculation in step S3, the point whose characteristic value is greater than the threshold value S is taken as the characteristic point, and the characteristic point of the template matching area M1 is recorded as , the feature points of the test matching area M2 are recorded as .

[0016] Preferably, the value of the threshold S is 25.

[0017] Preferably, after the feature points of the template matching area M1 and the test matching area M2 are obtained in step S3, the feature points of the two are compared by the least square method to complete the posture correction alignment of the image to be tested A relative to the template image M.

[0018] Preferably, in the least squares method used in step S3, first, the normal vector of the feature point D is calculated. ; Secondly, find the optimal translation and rotation transformation matrix , so that the image A to be tested passes through After translation and rotation, it is aligned with the template image M, that is, and exist The sum of squared distances in the direction is minimal; the optimal translation and rotation transformation matrix After determination, the coordinate data matrix corresponding to the image to be tested A is multiplied to obtain the adjustment image Ar after the posture correction and alignment of the image to be tested A relative to the template image M.

[0019] Preferably, the transformation matrix The calculation formula is

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] in, Image feature points of the test matching area M2; Image feature points of template matching area M1; for The normal vector of is the translation and rotation transformation matrix.

[0027] Preferably, the range binarization calculation in step S3 generates upper and lower limit images of the range binarization by shrinking and expanding the edge and increasing and decreasing the pixel grayscale value, wherein the edge shrinkage is to shrink the dark area in the template image T by one pixel inward through corrosion to form Tmin, and the edge expansion is to expand the dark area in the template image T by one pixel outward through expansion to form Tmax; then Tmin plus the grayscale value 30 is used as the upper limit image Tup of the range binarization; Tmax minus the grayscale value 30 is used as the lower limit image Tlow of the range binarization.

[0028] Preferably, when the adjustment image Ar is range-binarized with the upper limit image Tup and the lower limit image Tlow to form a mask image Amask, the pixels in the adjustment image Ar within the range of the upper limit image Tup and the lower limit image Tlow are marked as 0, and the pixels outside the range of TupTup and the lower limit image Tlow are marked as 255 to form a mask image Amask. The pixels with a value of 255 in the mask image Amask are where there are differences between Ar and the template image. The mask image Amask and the mask image Tm are then image-and-calculated to obtain a difference image E between the adjustment image Ar and the template image T in the large pattern area. The calculation formula of the difference image E is:

[0029] ;

[0030] .

[0031] Preferably, the area threshold C is determined by the minimum defect size to be detected. When the minimum defect to be detected is 1 square millimeter, and the millimeter pixel ratio of the camera is 0.2 millimeters / pixel, 1 square millimeter contains (1 / 0.2) 2 =25 pixels; since the defect is not necessarily a regular rectangle, the area threshold is multiplied by a coefficient of 0.8 on the basis of 25 pixels to obtain 20 pixels, which is the area threshold C; the area blocks in the difference map E that are larger than the area threshold C are further aligned by secondary template matching based on the pixel gray value, and then the range binarization calculation is performed again to obtain the final matching error. If it is still larger than the area threshold C value, it is judged as a real defect.

[0032] Preferably, in the small pattern area detection process of the image to be tested in step S4, first, the small pattern area is marked in the template image, and multiple images to be tested are collected and input into the program for running. During the program running process, the image to be tested and the template image are adjusted and corrected in posture to align the image to be tested with the template image; then, the test image is calibrated according to the small pattern area of ​​the template image to form local image pairs corresponding to each small pattern area, and the number of the local image pairs is the same as the number of the marked small pattern areas, and each small pattern area corresponds to a group of local image pairs; then, the local image pairs are data-labeled to form positive samples and negative samples; finally, the positive and negative samples are input into a deep learning change detection model for training to obtain a defect detection model for the small pattern area; the image to be tested collected in real time is input into the above-mentioned defect detection model, and the defect detection model performs detection and judgment, and outputs the judgment result.

[0033] The beneficial effects of the present invention are:

[0034] In view of the defects and shortcomings of the prior art, the present invention independently develops and designs a distorted cardboard defect detection method that uses independent detection methods for large pattern areas and small pattern areas to perform detection and judgment, effectively improving the accuracy of defect detection and improving detection efficiency.

[0035] The present invention aims to provide a machine detection method for realizing surface defect detection of distorted cardboard in an automated production process, which is applied to the field of cardboard production. Through this method, detection automation is realized, and detection accuracy and efficiency are improved. Specifically, the present invention processes large pattern parts and small text pattern parts in a cardboard image separately, matches, locates and detects large patterns according to a shape feature template matching method, and solves the processing of local distortion after matching through secondary matching based on pixel values. For small text patterns, due to distortion and inconsistency of the printing process, many small defects and special situations such as misalignment may occur. Traditional secondary matching is very slow when the number of defects is large, and it cannot solve special situations. Therefore, for small text patterns, this solution adopts change detection based on deep learning to process. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a block diagram of the detection principle of the present invention.

[0037] Figure 2 This is a block diagram of the detection principle of the present invention for a large pattern area.

[0038] Figure 3 This is a block diagram of the detection principle of the present invention for small pattern areas.

[0039] Figure 4 It is a schematic diagram of selecting a template matching area in a template image according to the present invention.

[0040] Figure 5 It is a schematic diagram of selecting a small pattern area in a template image according to the present invention.

[0041] Figure 6 It is a schematic diagram of the mask image corresponding to the small pattern area on the template image of the present invention.

[0042] Figure 7 Schematic diagram of the image to be tested according to the present invention.

[0043] Figure 8 It is a schematic diagram of the adjustment graph after the posture of the image to be tested is adjusted according to the present invention.

[0044] Fig. 9 It is a schematic diagram of the template image of the present invention after the edge is reduced.

[0045] Fig.10 It is a schematic diagram of the enlarged edge of the template diagram of the present invention.

[0046] Fig.11 for Fig. 9 Schematic diagram of the upper limit image formed after adding pixel values.

[0047] Fig.12 for Fig.10 Schematic diagram of the lower limit image formed after adding pixel values.

[0048] Fig.13 It is a difference map obtained after image calculation is performed between the mask image and the mask image of the present invention.

[0049] Fig.14 It is a schematic diagram of defect determination by area threshold in a difference map according to the present invention.

[0050] Fig.15 Schematic diagram of four examples of defects of the present invention.

[0051] Fig.16 It is a schematic diagram of defect marking on an image pair according to the present invention.

[0052] Fig.17 This is a schematic diagram of the deep learning change detection model of the present invention.

[0053] Fig.18 It is a schematic diagram of a data representation corresponding to the image A to be tested according to the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that all directional indications such as up, down, left, right, front, back, etc. in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0056] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0057] like Figure 1 As shown, a method for detecting deformed cardboard defects includes the following steps:

[0058] S1. Acquiring a template image: capturing a defect-free product image as a template image;

[0059] S2, collecting the image to be tested: photographing and collecting the image to be tested;

[0060] S3, large pattern area detection of the image to be tested: after the large pattern area is selected in the image to be tested acquired in step S2, the large pattern area is detected by matching the template image in step S1;

[0061] S4, small pattern area detection of the image to be tested: after selecting the small pattern area in the image to be tested collected in step S2, the template image in step S1 is matched to perform small pattern area detection;

[0062] S5, outputting the determination result: when the detection results outputted in step S3 and step S4 are both defect-free, the image to be tested is determined to be defect-free; when the detection results outputted in step S3 and / or step S4 are defective, the image to be tested is determined to be defective.

[0063] The present application proposes a method for detecting defects in distorted cardboard, which performs defect detection on cardboard images with local distortion. In general, the large pattern portion and the small text pattern portion in the cardboard image are detected and processed separately; the large pattern is matched, located and detected according to the shape feature template matching method, and the processing of the local distortion after matching is solved by secondary matching based on pixel values. For small text patterns, due to distortion and the inconsistency of the printing process, many small defects and special cases such as misalignment will occur. Traditional secondary matching is very slow when the number of defects is large, and it cannot solve special cases; therefore, for small text patterns, the present application uses change detection based on deep learning to process them. Example 1

[0064] like Figure 2 As shown, as an embodiment of the present invention, in the process of detecting the large pattern area in step S3 of the present invention, firstly, a template matching area M1 is selected on the template image T, and the feature values ​​of the pixels in the template matching area M1 are calculated to form edge shape features; in addition, as Figure 5 As shown in the figure, after marking the small pattern area in the template image T, the following is obtained by calculation: Figure 6 The mask image Tm shown; secondly, Figure 7As shown in FIG. 1 , after the selected test matching area M2 corresponding to the acquisition image A is obtained and the feature values ​​of the pixel points in the test matching area M2 are calculated, the edge shape features are formed; then, by comparing the feature points of the image A to be tested with those of the template image T, the posture adjustment of the image A to be tested is completed, and the following is obtained: Figure 8 The adjustment image Ar shown; after the template image T is respectively formed into an upper limit image Tup and a lower limit image Tlow by range binarization calculation, the adjustment image Ar, the upper limit image Tup and the lower limit image Tlow are range binarized to form a mask image Amask; the mask image Amask and the mask image Tm are image calculated to obtain a difference image E; in the difference image E, defect detection is performed through the area threshold C to determine the defects of the large pattern part of the image A to be tested, and the determination result is output.

[0065] In addition, if Figure 4 As shown, in this embodiment, when selecting a template matching area on the template image T, since the image is distorted more in the areas near the two sides, the selected template matching area should be close to the middle area of ​​the entire image, and an area containing special graphics should be selected to avoid template matching failure or matching multiple candidate areas. According to actual experience, the middle area accounts for about 60% of the image width.

[0066] In addition, if Figure 5 As shown in the figure, for the template image T, the basic color of the cardboard (the basic color value of the yellow cardboard in this example is (R:107, G:96, B:36)) is used as a reference, and the pixels in the image whose pixel values ​​are within a certain range from the basic value (the difference between the value of each RGB component and the basic value is less than 30) are found to obtain the mask image Tm of the cardboard area in the template image T, and the areas with large distortion on both sides and the small text areas are removed. The mask image is subsequently used to perform shape feature template matching defect detection on the large pattern part. Example 2

[0067] As an embodiment of the present invention, when calculating the characteristic values ​​of the pixels in the template matching area M1 or the test matching area M2 in step S3 of the present invention, a convolution calculation method is adopted, that is, the pixel values ​​of the pixels in the template matching area M1 or the test matching area M2 are weighted and summed. The calculation process is to multiply each pixel value by each element of the convolution template respectively, and sum the result of the convolution. The sum calculated is the result of the convolution operation. The characteristic value calculation formula is as follows:

[0068]

[0069] Among them, g(i,j) is the calculated eigenvalue corresponding to the pixel point, f is the image matrix, (i,j) is the coordinate of f, h is the convolution template, and (k,l) is the coordinate of h.

[0070] After the feature value is obtained by calculation in step S3, the point whose feature value is greater than the threshold S is taken as the feature point, and the feature point of the template matching area M1 is recorded as , the feature points of the test matching area M2 are recorded as .

[0071] In this embodiment, the template image and the image to be tested are yellow cardboard, and the threshold S is selected as 25 according to its background color characteristics; in addition, the threshold S for cardboards with different background colors can be adjusted to a certain extent. The principle is that the greater the difference between the background color and the foreground color, the larger the threshold S is, and vice versa.

[0072] After the feature points of the template matching area M1 and the test matching area M2 are obtained in step S3, the feature points of the two are compared by the least square method to complete the posture correction alignment of the image to be tested A relative to the template image M.

[0073] In the least squares method used in step S3, first, the normal vector of the feature point D is calculated. ; Secondly, find the optimal translation and rotation transformation matrix , so that the image A to be tested passes through After translation and rotation, it is aligned with the template image M, that is, and exist The sum of squared distances in the direction is minimal; the optimal translation and rotation transformation matrix After determination, the coordinate data matrix corresponding to the image to be tested A is multiplied to obtain the adjustment image Ar after the posture correction and alignment of the image to be tested A relative to the template image M.

[0074] Furthermore, the coordinate data matrix corresponding to the image A to be tested is the data representation of the image A to be tested. For example, if the image A is an image with a width of M pixels, a height of N pixels, and a channel number of D, then the matrix A is a three-dimensional matrix of M*N*D, where M and N correspond to the horizontal and vertical coordinate systems of the matrix, and the channel number D corresponds to the number of data at a certain coordinate of the matrix, such as Fig.18 The image shown here is a 3-channel color image with a width of 15 pixels and a height of 10 pixels. Its data representation is a 15*10*3 matrix with 3 data at each coordinate position.

[0075] Transformation Matrix The calculation formula is

[0076] ;

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] in, Image feature points of the test matching area M2; Image feature points of template matching area M1; for The normal vector of is the translation and rotation transformation matrix.

[0083] like Figures 9 and 10 As shown in FIG. 1 , since a certain calculation error is inevitably generated during the rotation and translation alignment process, the image actually aligned still has a slight error of at least 1 pixel at the edge of the pattern. Therefore, in order to reduce the impact of the error and improve the detection accuracy, the range binarization calculation in step S3 of the present invention generates the upper and lower limit images of the range binarization by shrinking and expanding the edge and increasing and decreasing the pixel grayscale value, wherein the edge shrinkage is to shrink the dark area in the template image T by one pixel inward by corrosion to form Tmin, and the edge expansion is to expand the dark area in the template image T by one pixel outward by expansion to form Tmax; as shown in FIG. Figure 11 to Figure 12 As shown, Tmin plus the grayscale value 30 is used as the upper limit image Tup of the range binarization; Tmax minus the grayscale value 30 is used as the lower limit image Tlow of the range binarization. The above value 30 is determined according to the background color of the sample in the actual detection process, and can be adjusted in real time according to different detection samples in actual application.

[0084] like Fig.13 As shown, when the adjustment image Ar is binarized with the upper limit image Tup and the lower limit image Tlow to form a mask image Amask, the pixels in the adjustment image Ar within the range of the upper limit image Tup and the lower limit image Tlow are marked as 0, and the pixels outside the range of TupTup and the lower limit image Tlow are marked as 255 to form a mask image Amask. The pixels with a value of 255 in the mask image Amask are where there are differences between Ar and the template image. The mask image Amask and the mask image Tm are then image-and-calculated to obtain a difference image E between the adjustment image Ar and the template image T in the large pattern area. The calculation formula of the difference image E is:

[0085] ;

[0086] .

[0087] After the difference map E is obtained, it is necessary to compare and determine each location in the difference map E using the area threshold C. The area threshold C is determined by the minimum defect size to be detected. When the minimum defect to be detected is 1 square millimeter, and the millimeter pixel ratio of the camera is 0.2 millimeters / pixel, then 1 square millimeter contains (1 / 0.2) 2 =25 pixels; since the defect is not necessarily a regular rectangle, the area threshold is multiplied by a coefficient of 0.8 on the basis of 25 pixels to obtain 20 pixels, which is the area threshold C; for each difference local image of the difference map E, the area block greater than the area threshold C is further aligned by secondary template matching based on the pixel gray value, and then the range binarization calculation is performed again to obtain the final matching error. If it is still greater than the area threshold C value, it is judged as a real defect; the above-mentioned secondary template matching alignment method and range binarization calculation process for the difference map E are the same as the matching posture adjustment alignment method of the template map T and the image to be tested A and the method of binarization calculation of the adjustment map Ar and the upper and lower images Tup and the lower limit image Tlow in the previous text, so they are not elaborated here.

[0088] Further, such as Fig.14 The figure shows the result of the secondary matching verification of the three differences in the difference map E whose area is larger than the threshold. Taking defect a as an example, the defect is expanded by 10 pixels, and the corresponding part of defect a in T is taken as at, and the corresponding part in Ar is taken as ad. The pixel gray value template matching method matchTemplate provided by opencv is used to perform secondary matching alignment of at and ad (the reason why feature point matching is not used is that the defect area here is small, and the matching speed using pixel gray value will not be very slow, and the calculation is relatively convenient). After alignment, the range binarization method used above is used again to calculate the error of aligned at and ad to obtain ae. In this case, the area of ​​white pixels in ae is still larger than the area threshold C, so defect a is a real defect, not a false defect caused by alignment error. On the contrary, defects b and c are false defects caused by the matching alignment error between Ar and T. There are almost no white pixels in the error maps be and ce calculated after secondary matching alignment; therefore, defects b and c are eliminated, and the real defect is finally determined to be a. Example 3

[0089] like Figure 3As shown, the detection method for small pattern areas of the present invention is different from the detection method for large pattern areas. In the small pattern area detection process of the image to be tested in step S4, the small pattern area is first marked in the template image, and multiple images to be tested are collected and input into the program for running. During the program running process, the image to be tested and the template image are adjusted and corrected in posture to align the image to be tested with the template image; then, the test image is calibrated according to the small pattern area of ​​the template image to form local image pairs corresponding to each small pattern area. The number of the local image pairs is the same as the number of the marked small pattern areas, and each small pattern area corresponds to a group of local image pairs; then, the local image pairs are data-marked to form positive samples and negative samples; finally, the positive and negative sample input values ​​are trained in the deep learning change detection model to obtain a defect detection model for the small pattern area; the image to be tested collected in real time is input into the above-mentioned defect detection model, and the defect detection model performs detection and judgment, and outputs the judgment result.

[0090] like Fig.15 As shown in the figure, four common examples are shown. For the detection of small text pattern areas, the template matching alignment method to find differences is invalid due to some special reasons. Specifically, there are the following situations: First, due to the order of printing, there is often a misalignment deviation between text patterns; second, due to the deviation of the imaging position, the image brightness changes greatly, which is easy to exceed the threshold of the range binarization; third, due to the overall or local deformation caused by distortion, including rotation, scaling, stretching, etc.; fourth, some defects such as missing printing of small strokes and small ink spots on paper cannot be detected by the area threshold due to their small area. It is impossible to completely align the above four situations if template matching is performed for the text pattern together. One, two, and three are normal and acceptable, but the algorithm will make false detections, and the fourth is that the algorithm will miss detections.

[0091] Furthermore, for the case of small text pattern areas, this solution adopts a change detection method based on deep learning. First, a large number of actual test images are run through the program, and local image pairs after template matching calibration are collected according to the pre-marked small text pattern frames, such as Fig.16 The similar partial image pairs are listed as shown in the figure. The template image is marked as BTi, and the test image is marked as BAi, i=[1,N], N is the number of framed text areas, BTi and BAi are a pair of images; then the collected image pairs are labeled, and the data labels are divided into positive samples and negative samples. Positive samples are acceptable. Fig.15The image pairs with misalignment, brightness change, rotation, scaling, and stretching are acceptable. These image pairs are positive samples and are placed in the positive sample folder. The image pair files are named XXXX-0 and XXXX-1, where XXXX is a sequence number representing different image pairs, and 0 and 1 correspond to the two images of the image pair. Positive samples have no defects and do not need to be marked on the image. Negative samples are image pairs with real defects, such as Fig.15 5 sets of image pairs with small defects are placed in the negative sample folder. The naming is similar to the positive sample. XXXX-0 in the image pair corresponds to the normal image, XXXX-1 corresponds to the defective image, and the location of the defect should be marked with color on XXXX-1, such as Fig.16 Shown are examples of defect-marked image pairs.

[0092] Finally, the deep learning change detection model is trained to obtain a defect detection model for small text pattern areas. The structural design of the change detection model used in this example is as follows: Fig.17 In actual use, the model is used to infer and recognize the text pattern area image pairs of the template image and the test image, as shown in Fig.17 Input a pair of images as shown, and give an inference result, 0 represents no defect, 1 represents defect, and if there is a defect, a defect mask image will be given, such as Fig.17 Output is shown; In this way, defect detection in small text pattern areas is achieved, and the recognition rate is much higher than that of traditional template matching methods, and the false detection rate is greatly reduced. In addition, the deep learning detection model belongs to the existing technology, so it will not be described in detail in this application.

[0093] The embodiments of the present invention are only to introduce its specific implementation methods and are not intended to limit its protection scope. The technicians in this industry can make some modifications inspired by this embodiment, so any equivalent changes or modifications made according to the scope of the patent of the present invention are within the scope of the patent claims of the present invention.

Claims

1. A method for detecting deformed cardboard defects, characterized in that: The steps include: S1. Acquiring a template image: capturing a defect-free product image as a template image; S2, collecting the image to be tested: photographing and collecting the image to be tested; S3, large pattern area detection of the image to be tested: after the large pattern area is selected in the image to be tested acquired in step S2, the large pattern area is detected by matching the template image in step S1; S4, small pattern area detection of the image to be tested: after selecting the small pattern area in the image to be tested collected in step S2, the template image in step S1 is matched to perform small pattern area detection; S5, outputting the determination result: when the detection results outputted in step S3 and step S4 are both defect-free, it is determined that the image to be tested is defect-free; when the detection results outputted in step S3 and / or step S4 are defective, it is determined that the image to be tested is defective; In the process of detecting the large pattern area in step S3, firstly, a template matching area M1 is selected on the template image T, and the feature values ​​of the pixels in the template matching area M1 are calculated to form edge shape features; in addition, a mask image Tm is obtained after marking the small pattern area in the template image T; Secondly, after collecting the corresponding selected test matching area M2 in the image A and calculating the feature values ​​of the pixels in the test matching area M2, the edge shape feature is formed; Then, by comparing the feature points of the image A to be tested with the template image T, the posture adjustment of the image A to be tested is completed to obtain the adjustment image Ar; After the template image T is range-binarized to form an upper limit image Tup and a lower limit image Tlow, the adjustment image Ar is range-binarized with the upper limit image Tup and the lower limit image Tlow to form a mask image Amask; Performing image calculation on the mask image Amask and the mask image Tm to obtain a difference image E; In the difference image E, defect detection is performed by using the area threshold C to determine the defects of the large pattern part of the image to be tested A, and the determination result is output; When calculating the characteristic values ​​of the pixels in the template matching area M1 or the test matching area M2 in step S3, a convolution calculation method is used, that is, the pixel values ​​of the pixels in the template matching area M1 or the test matching area M2 are weighted and summed. The calculation process is to multiply each pixel value by each element of the convolution template, and sum the convolution results. The sum calculated is the result of the convolution operation. The characteristic value calculation formula is as follows: ; Among them, g(i,j) is the calculated eigenvalue corresponding to the pixel point, f is the image matrix, (i,j) is the coordinate of f, h is the convolution template, and (k,l) is the coordinate of h; After the characteristic value is obtained by calculation in step S3, the point whose characteristic value is greater than the threshold S is taken as the characteristic point, and the characteristic point of the template matching area M1 is recorded as , the feature points of the test matching area M2 are recorded as ; After the feature points of the template matching area M1 and the test matching area M2 are obtained in step S3, the feature points of the two are compared by the least square method to complete the posture correction alignment of the image to be tested A relative to the template image M; In the least squares method used in step S3, first, the normal vector of the feature point D is calculated. ; Secondly, find the optimal translation and rotation transformation matrix , so that the image A to be tested passes through After translation and rotation, it is aligned with the template image M, that is, and exist The sum of squared distances in the direction is minimal; the optimal translation and rotation transformation matrix After determination, the coordinate data matrix corresponding to the image to be tested A is multiplied to obtain the adjustment image Ar after the posture correction and alignment of the image to be tested A relative to the template image M.

2. The method for detecting deformed paperboard defects according to claim 1, characterized in that: The value of the threshold S is 25.

3. The method for detecting deformed paperboard defects according to claim 1, characterized in that: The range binarization calculation in step S3 generates upper and lower limit images of the range binarization by shrinking and expanding the edge and increasing and decreasing the pixel grayscale value, wherein the edge shrinkage is to shrink the dark area in the template image T by one pixel inward through corrosion to form Tmin, and the edge expansion is to expand the dark area in the template image T by one pixel outward through expansion to form Tmax; then Tmin plus the grayscale value 30 is used as the upper limit image Tup of the range binarization; Tmax minus the grayscale value 30 is used as the lower limit image Tlow of the range binarization.

4. The method for detecting deformed paperboard defects according to claim 3, characterized in that: When the adjustment image Ar is binarized with the upper limit image Tup and the lower limit image Tlow to form a mask image Amask, the pixels in the adjustment image Ar within the range of the upper limit image Tup and the lower limit image Tlow are marked as 0, and the pixels outside the range of Tup and the lower limit image Tlow are marked as 255 to form a mask image Amask. The pixels with a value of 255 in the mask image Amask are where there are differences between Ar and the template image. The mask image Amask and the mask image Tm are then image-and-calculated to obtain a difference image E between the adjustment image Ar and the template image T in the large pattern area. The calculation formula of the difference image E is: ； 。 5. The method for detecting deformed paperboard defects according to claim 4, characterized in that: The area threshold C is determined by the minimum defect size to be detected. When the minimum defect to be detected is 1 square millimeter and the millimeter pixel ratio of the camera is 0.2 millimeters / pixel, 1 square millimeter contains (1 / 0.2) 2 =25 pixels; since the defect is not necessarily a regular rectangle, the area threshold is multiplied by a coefficient of 0.8 on the basis of 25 pixels to obtain 20 pixels, which is the area threshold C; the area blocks in the difference map E that are larger than the area threshold C are further aligned by secondary template matching based on the pixel gray value, and then the range binarization calculation is performed again to obtain the final matching error. If it is still larger than the area threshold C value, it is judged as a real defect.

6. The method for detecting deformed paperboard defects according to claim 1, characterized in that: In the process of detecting the small pattern area of ​​the image to be tested in step S4, firstly, the small pattern area is marked in the template image, and multiple images to be tested are collected and input into the program for running. During the program running process, the image to be tested and the template image are adjusted and corrected in position and posture so that the image to be tested and the template image are aligned; Then, the test image is calibrated according to the small pattern area of ​​the template image to form local image pairs corresponding to each small pattern area. The number of the local image pairs is the same as the number of the marked small pattern areas, and each small pattern area corresponds to a group of local image pairs; then, the local image pairs are data labeled to form positive samples and negative samples; finally, the positive samples and negative samples are input into the deep learning change detection model for training to obtain a defect detection model for the small pattern area; the image to be tested collected in real time is input into the above-mentioned defect detection model, and the defect detection model performs detection and judgment, and outputs the judgment result.

Images