A visual inspection method based on sequence image difference

Abstract

The application discloses a kind of sequence image difference part surface defect detection methods, specific method as follows:First, selecting sequence image in same production batch, the registration of adjacent parts before and after is carried out;Second, in the process of image registration, sequence difference needs to be carried out to part image;Finally, it is judged whether the part to be measured is qualified or not.It is gradually accumulated due to the error in the process of silk screen printing, so when the production sequence difference between template parts and detection parts is too large, accurate difference cannot be achieved, so the above-mentioned sequence image difference part surface defect detection method is used to realize high-precision, high-stability detection of surface feature complex part printing quality defects based on visual detection technology.

Description

Technical Field

[0001] This invention relates to the field of precision geometric measurement technology based on machine vision, and proposes a visual detection method based on sequence image difference to address the deformation of printed images caused by the stretching deformation of the screen during screen printing. Background Technology

[0002] In today's era of economic globalization driving the deepening development of import and export trade, domestic and foreign enterprises have set stringent procurement standards for the surface quality of parts widely used in various industries. Achieving accurate and stable detection of surface quality defects in parts is of profound significance for enterprises to improve production quality, increase production efficiency, expand domestic and international markets, and promote industrial and technological upgrading and innovation. Meanwhile, screen printing, with its advantages of large batch sizes, low prices, vibrant colors, and fast delivery, is increasingly recognized by various industries and has a wide range of applications, including circuit boards in household appliances, patterns on textiles, designs on shoes, and text on the panels of refrigerators, televisions, and washing machines. Therefore, visual inspection of surface defects in sequential parts is urgently needed.

[0003] In recent years, research teams and scholars worldwide have achieved significant results in the research of surface damage detection technology for parts. For example, the Hogeveen steel plant in Imeiden, Netherlands, has built an online real-time classification system for detecting surface defects in steel strips. This system can detect defects as small as 0.85mm × 0.60mm and classify at least 50 different types of defects. Meanwhile, Zhang Kai from the University of Electronic Science and Technology of China has researched surface crack defects in industrial products, using infrared thermal imaging non-destructive testing technology based on line laser scanning excitation, and has built a laser scanning infrared thermal imaging non-destructive testing system with adjustable optical paths.

[0004] In summary, there are many technologies applied to surface defect detection. However, when most teams and experts study such problems, according to the research ideas, they can be mainly divided into three methods: image segmentation method, template matching method, and laser scanning method. Among them, the accuracy and robustness of the image segmentation method are difficult to meet the requirements, there is a phenomenon of missed detection, it is difficult to adapt to complex environments, and it is impossible to accurately identify sub-pixel level defects. Although the laser scanning method has high accuracy and high stability, it has the disadvantages of high equipment price, easy reflection, inability to obtain detection information in real time, and being greatly affected by the part surface. Moreover, it cannot detect the printed images on the part surface. Therefore, the present invention conducts a more in-depth exploration of the feature-based template matching method. Through experimental research, it is found that during the screen printing process, the screen plate has a certain degree of stretching deformation, and the magnitude of the printing screen tension directly affects the printing accuracy and quality, and at the same time affects the accuracy of the detection result. If the same template is selected and differential is performed on all parts, due to the deformation of the template, the images cannot be accurately differentiated during the differential process. When the production order of the template part and the detected part is relatively large, during the differential process, the result after differential will not be obvious. Therefore, this article proposes to perform differential on sequential images. Summary of the Invention

[0005] The object of the present invention is to provide a visual detection method based on sequential image differential, so as to achieve high-precision and high-stability detection of the printing quality defects of parts with complex surface features based on visual detection technology.

[0006] To achieve the above object, the present invention provides a visual detection method based on sequential image differential, and the specific method is as follows:

[0007] S1. Select sequential images in the same production batch, and perform differential on adjacent parts before and after.

[0008] S2. During the process of performing differential on the image sequence, it is necessary to perform image registration on the part images selected in step 1.

[0009] S3. Judge whether the part to be detected is qualified.

[0010] Preferably, during the process of image sequence differential in step S1, the gray value G of the standard part < K, where G is the gray value of the standard part, K is the gray value of each point under different working conditions, and the gray value K has uniformity.

[0011] Preferably, in step S2, the image registration process includes feature region extraction and image sub-pixel subdivision.

[0012] The line feature extraction is to record the straight-line distance between two center points as D, and represent the entire image through a feature center line, converting the registration of the two images from the registration problem between the original feature points into the registration of the connection lines between the feature points.

[0013] The image subpixel subdivision method is based on quadratic bilinear interpolation. Within the existing algorithm framework, it uses known point information to infer unknown point information, dividing a pixel into four equal parts in both the X and Y directions, thus dividing the original one pixel into 16 equal pixels. The formula is as follows:

[0014]

[0015] Where PR represents physical resolution, in mm / pixel; SS represents the length of the part under test in a single direction, in mm; SDR represents the image resolution of the part under test in a single direction, in pixels; and NI represents the number of image interpolations, in times.

[0016] Preferably, the process for image sequence differencing is as follows:

[0017] (1) Perform image registration and difference between the part to be inspected 1 and the template part 1, G S The grayscale value of a certain point on the part to be tested after differential analysis is used to determine whether the initial inspection result is qualified.

[0018] (2) If G S If -K3≤0, it means that the gray value of each point in the image is less than the gray value of the selected standard, then the part is qualified, and the part to be inspected 1 is marked as template part 2;

[0019] (3) If G S If -K3>0, then there is a gray value in the image that is greater than the selected standard gray value, and the part is unqualified. Thus, it is determined whether the surface D value of the part to be inspected is within the acceptable range, where D is the distance between feature points on the surface of the part.

[0020] (4)D M D represents the distance between feature points of the template part. S Let |D be the distance between feature points of the part to be measured. S -D M If |≤E, it means the two images match correctly, where E is the maximum allowed distance, but G... S -K3>0, so it is recorded as a defective product;

[0021] (5) If |D S -D M If |>E, an alarm will be triggered, and a second manual re-inspection will be performed;

[0022] (6) If the manual re-inspection is qualified, it is template part 2; if the manual re-inspection is unqualified, it is judged as a defective product.

[0023] Preferably, the image sequence difference determination results fall into the following categories:

[0024] Scenario 1: Perform registration differential between template part 1 and part 1 to be inspected. If G S If -K>0, the initial test fails, but |D S -D M If E is detected, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection is successful, the part to be inspected 1 will be regarded as template part 2.

[0025] Scenario 2: Perform registration differential between the part to be inspected (2) and the previous qualified part (i.e., template part 2), which is the part to be inspected (1). If G S If -K>0, the initial test fails, but |D S -D M If the result is E, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection fails, the product will be deemed defective.

[0026] Scenario 3: Perform registration differential between the part to be inspected (3) and the previous qualified part (i.e., template part 2), which is the part to be inspected (1). If G S If -K>0, the initial test fails, and |D S -D M If |≤E, no manual re-inspection is required; it is directly judged as a defective product.

[0027] Scenario 4: Perform registration differential between the part to be inspected (4) and the previous qualified part (i.e., template part 2), which is also the part to be inspected (1). If G S If -K≤0, the initial inspection is qualified, and the part to be inspected 4 will be used as the template part 3 for subsequent inspections.

[0028] Therefore, the present invention employs the above-mentioned method for detecting surface defects of parts by sequential image difference. After image registration, it can effectively solve the interference of shadow error caused by the deformation of the template itself. Compared with the traditional difference method, it is more effective in eliminating shadow error interference, which verifies the feasibility of the method and provides a strong guarantee for the entire process of detecting surface printing quality defects.

[0029] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0030] Figure 1 This is a flowchart of the detection process of a part surface defect detection method based on sequence image difference according to the present invention;

[0031] Figure 2 This is a schematic diagram of the surface feature regions, feature points, and feature point spacing of a part surface in a part surface defect detection method based on sequence image difference according to the present invention.

[0032] Figure 3This is a graph showing the variation of the feature point spacing D value of standard parts in the same batch, based on the part surface defect detection method of sequential image difference according to the present invention.

[0033] Figure 4 This is a graph showing the variation of the Z-value of the feature point spacing difference in the same batch of standard parts in the part surface defect detection method of the present invention, which is based on sequential image difference.

[0034] Figure 5 This invention relates to a method for detecting surface defects in parts using sequential image difference; (The image shows a surface-printed marking image.)

[0035] Figure 6 This is a detection result diagram of the registration difference between qualified and unqualified parts in a part surface defect detection method of sequential image difference according to the present invention, wherein (A) is the detection result of the registration difference between qualified part 13 and unqualified part 18, and (B) is the detection result of the registration difference between qualified part 17 and unqualified part 18.

[0036] Figure 7 This is a comparison diagram of the results of four groups of traditional difference and image sequence difference methods for the part surface defect detection method of sequential image difference according to the present invention, wherein (A) is the first group, (B) is the second group, (C) is the third group, and (D) is the fourth group.

[0037] Figure 8 This is a schematic diagram of the spatial positional relationship between the two center lines of a part surface defect detection method based on sequence image difference according to the present invention.

[0038] Figure 9 This is a spatial translation schematic diagram of a part surface defect detection method based on sequence image difference according to the present invention;

[0039] Figure 10 This invention relates to a spatial rotation process in a method for detecting surface defects in parts using sequence image difference. Detailed Implementation

[0040] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] A method for detecting surface defects in parts using sequential image differentiation is disclosed. In this method, sequential images refer to adjacent parts from the same production batch that are differentiated. This method primarily considers the situation where the error variation between adjacent templates is relatively small as production time changes.

[0042] During the process of image sequence difference, parts with a gray value G > K1 after image difference have surface defects, those with G < K2 are background noises, and the standard parts selected in this study require G < K3. The values of K1, K2, and K3 are the gray values at each point under different working conditions. At the same time, the gray value requirements are uniform and there are no sudden changes. This experiment has very strict requirements for the selection of standard parts.

[0043] The flow chart of the detection process is as Figure 1 shown. Among them, the four existing situations are represented by thick solid lines, which completely express the loop process of the flow chart:

[0044] Perform image registration difference on the part to be detected 1 and the template part 1. G S (X i , Y j ) is the gray value of a certain point of the part to be detected after difference, where i and j are the horizontal and vertical coordinates of the image. Judge whether the initial inspection result is qualified;

[0045] If G S - K3 ≤ 0, it means that the gray value of each point in the picture is less than the gray value of the selected standard, so the part is qualified, and mark the part to be detected 1 as the template part 2;

[0046] If G S - K3 > 0, there are gray values in the picture that are greater than the selected standard gray value, so the part is unqualified. Then judge whether the D value on the surface of the part to be detected 1 is within the acceptable range;

[0047] The coordinates of feature point 1 are D A (x1, y1); the coordinates of feature point 2 are D B (x2, y2), as Figure 2 shown;

[0048] where D M is the distance between the feature points of the template part, D S is the distance between the feature points of the part to be detected. If |D​​​​​​​​​​​​​If the manual re-inspection is qualified, it is considered template part 2; if the manual re-inspection is unqualified, it is judged as a defective product. The above cycle repeats for the four situations.

[0051] Scenario 1: Perform registration differential between template part 1 and part 1 to be inspected. If G S If -K3>0, the initial test fails, but |D S -D M If E is detected, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection is successful, the part to be inspected, 1, will be regarded as template part 2.

[0052] Scenario 2: Perform registration differential between the part to be inspected (2) and the previous qualified part, i.e., template part 2 (part to be inspected 1). If G S If -K3>0, the initial test fails, but |D S -D M If the result is E, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection fails, the product will be deemed defective.

[0053] Scenario 3: Perform registration differential between the part to be inspected (3) and the previous qualified part, i.e., template part 2 (part to be inspected 1). If G S If -K3>0, the initial test fails; simultaneously |D S -D M If |≤E, no manual re-inspection is required; it is directly judged as a defective product.

[0054] Scenario 4: Perform registration differential between the part to be inspected (4) and the previous qualified part, i.e., template part 2 (part to be inspected 1). If G S If -K3≤0, the initial inspection is qualified, and the part to be inspected, 4, will be used as the template part 3 for subsequent inspections.

[0055] Meanwhile, in the actual inspection process, image registration is also required for the captured part images. Image registration can effectively reduce the error generated by the edge of the part during the difference of sequential images and more accurately distinguish the defect area. The image registration process includes two parts: feature region extraction and image sub-pixel subdivision.

[0056] In line feature extraction, the straight-line distance between two center points is denoted as D. A feature center line represents the entire image, transforming the registration of two images from a problem of registering feature points directly between them into registering the lines connecting those feature points. Feature regions, feature points, and the distance between feature points D are represented as follows: Figure 2 As shown.

[0057] like Figure 8 The diagram shows the spatial relationship between the two center lines. The extracted point and line features are used to complete feature matching between the two images. The registration mapping relationship between the images is established through the matching relationship of point and line features. The image spatial coordinate transformation parameters are obtained, and finally the registration between the two images is completed with the help of the coordinate transformation parameters.

[0058] like Figure 9 As shown, applying image space transformation to completely overlap and register two line segments is essentially an operation between two matrices. If the center points A1(x1, y1) and A3(x3, y3) of the same feature in two images are overlapped, then the coordinates of A3(x3, y3) and A4(x4, y4) after two translation transformations are A5(x5, y5) and A6(x6, y6).

[0059] Construct a triangle by connecting the two characteristic center lines with an auxiliary line. Use the Pythagorean theorem to calculate the distance between the two characteristic center lines and the auxiliary line c. The calculation formula is shown below:

[0060]

[0061]

[0062]

[0063] like Figure 10 As shown, the rotation angle θ is determined using the law of cosines, and then input into the image rotation algorithm to complete image registration. The stability of the template matching method based on line features will be verified subsequently, as shown in the following formula:

[0064]

[0065] The sub-pixel subdivision method in this study is mainly based on quadratic bilinear interpolation. Its principle is to use known point information to infer unknown point information within the existing algorithm framework, dividing a pixel into four equal parts in the X and Y directions, thus dividing the original one pixel into 16 pixels. The formula is shown below:

[0066]

[0067] Where PR represents physical resolution (unit: mm / pixel); SS represents the length of the part under test in a single direction (unit: mm); SDR represents the image resolution of the part under test in a single direction (unit: pixels); and NI represents the number of image interpolations (unit: times). This method improves image resolution while ensuring the operating efficiency of the detection system, reduces visual distortion caused by resizing the image to a non-integer scaling factor, minimizes the theoretical registration error introduced during image preprocessing, meets the requirements of actual working conditions, and lays the foundation for image registration.

[0068] Example

[0069] First, the reasons for the differences in the parts themselves are analyzed from the actual printing process of the parts. The variation law of printing templates and template deformation of the same batch of standard parts is explored. A defect detection method based on sequential image difference is proposed and its effectiveness and stability are verified. Finally, the offset error caused by the differences in the parts themselves is effectively eliminated. In the process of sequential difference, the images acquired by the camera also need to be registered. The registration is mainly performed by first using the line feature extraction template registration method for difference, and then by sub-pixel image subdivision. The images are registered accurately and efficiently, which can improve the efficiency and accuracy of the parts in the process of sequential image difference.

[0070] The surface pattern selected in this article was produced using screen printing in actual production. During the printing process, it was observed that the printing screen has a certain degree of elasticity. However, under long-term production line inspection conditions, the probability of screen fatigue increases with usage time, and it may experience tensile deformation to some extent. The tension of the printing screen directly affects the accuracy and quality of the printing.

[0071] This paper inspects 20 standard parts from the same batch, sorting them by production time from earliest to latest. The feature point spacing D value is used as a standard to determine the variation pattern of the part template. Figure 3 As shown; the difference Z value between feature points is used as a standard to determine the error variation pattern of different parts, such as... Figure 4 As shown; where the D value is mainly the distance between two feature points, such as Figure 2 As shown; simultaneously, the difference in the feature point spacing between different parts A and B with the same template is denoted as Z. AB =|D A -D B Theoretically, the difference Z for parts with the same template should be 0. In subsequent verification experiments, the presence of errors in the detection results can be quickly determined by comparing the magnitude of D and whether the magnitude of Z is 0.

[0072] Depend on Figure 3 , 4 It can be seen that as production time progresses, the template generally tends to increase in size and length, but its randomness is relatively strong and the template error changes without obvious regularity. Therefore, in order to address the part difference problem caused by fatigue tensile deformation of the printing screen itself, a sequence image difference method is proposed for exploration. At the same time, in order to achieve stable detection of surface defects of metal parts, based on the above ideas, the algorithm is improved on the basis of feature point matching, and a template matching method based on line feature extraction is adopted.

[0073] After improvement, the typical printed pattern on the surface of the part in the image is selected as the feature area by the template matching method based on line features. By carefully observing the printed patterns on the surfaces of the parts to be measured with different templates, it is found that each type of printed pattern on the part surface has some regularly shaped printed identification patterns, such as Figure 5 As shown, the feature point registration problem is changed to the feature line registration problem. Its main advantage is that once the position of the part changes slightly due to positioning factors during the image acquisition process or the coordinate information of the extracted points changes due to certain environmental factors, it directly causes the spatial position coordinates of the points to change accordingly. However, in this case, since the distance D between two points, that is, the length of the feature line, remains unchanged, so compared with others, using the feature line distance D as a feature for registration is a better method.

[0074] In this paper, to explore the accuracy and robustness of image sequence difference, three parts numbered 13, 17, and 18 are selected from the parts for verification. Among them, parts numbered 13 and 17 are qualified parts, and part numbered 18 is a defective part. The numbers are used to simulate the order of parts on the production line during the actual detection process. The registration difference detection results of the qualified part 13 and the unqualified part 18 are as shown in Figure 6 Figure (A) in it. It is found that there are more error effects on defect recognition in addition to the detected defects in the figure. The registration difference detection results of the qualified part 17 and the unqualified part 18 are as shown in Figure 6 Figure (B) in it. It is found that the defects to be detected can be effectively recognized and there will be no serious error interference. Comparing the experimental results of Figure (A) and Figure (B), it is found that the defect detection method based on sequence image difference proposed above has a good detection effect and realizes the effective recognition of the defects to be measured.

[0075] To verify the accuracy of the above experiment, this paper also conducts a certain degree of batch experiment to fully analyze the accuracy and reliability of image sequence difference. The detection results include: the difference results between non-adjacent standard parts and defective parts to be measured, that is, the traditional image difference defect detection method, and the difference results between adjacent standard parts and defective parts to be measured, that is, the defect detection method based on sequence image difference for comparison. As shown in Figure 7 As shown, they are partial diagrams of the comparison of four groups of traditional difference and image sequence difference results respectively. It can be clearly seen from the images that the defect detection method based on image sequence difference proposed in this study, after being paired with image registration, can effectively solve the shadow error interference introduced by template deformation due to the part itself. Compared with the traditional difference method, it has a better effect in eliminating shadow error interference, verifying the feasibility of this method. At the same time, the minimum detectable defect size is about 0.15mm×0.12mm, which greatly meets the actual defect detection requirements of the enterprise of 0.2mm×0.2mm, providing a strong guarantee for the entire surface printing quality defect detection process.

[0076] Therefore, the present invention employs the above-mentioned method for detecting surface defects of parts by sequential image difference, to achieve high-precision and high-stability detection of printing quality defects of parts with complex surface features based on visual inspection technology.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A visual detection method based on sequence image difference, characterized in that, The specific method is as follows: S1. Select the sequence images in the same production batch, and perform differencing on the adjacent parts before and after; The gray value G of the standard part satisfies G < K, where G is the gray value of the standard part, K is the gray value of each point under different working conditions, and the gray value K has uniformity; S2. During the process of image sequence differencing, it is necessary to perform image registration on the part images selected in S1; Among them, the process of image sequence differencing is as follows: (1) Perform image registration and difference between the part to be inspected 1 and the template part 1, G S The grayscale value of a certain point on the part to be tested after differential analysis is used to determine whether the initial inspection result is qualified. (2) If G S -K≤0 means that the gray value of each point in the image is less than the gray value of the selected standard, then the part is qualified, and the part to be inspected 1 is marked as template part 2; (3) If G S If -K>0, then there is a gray value in the image that is greater than the selected standard gray value, and the part is unqualified. This determines whether the surface D value of the part to be inspected is within the acceptable range, where D is the straight-line distance between feature points on the surface of the part. (4) D M D is the straight-line distance between feature points of the template part. S Let |D| be the straight-line distance between feature points of the part to be measured. S -D M If |≤E, it means the two images match correctly, where E is the maximum allowed distance, but G... S -K>0, so it is recorded as a defective product; (5) If |D S -D M If |>E, an alarm will be triggered, and a second manual re-inspection will be performed. (6) If the manual re-inspection is qualified, it is the template part 2; if the manual re-inspection is unqualified, it is determined as a defective product; The judgment results of image sequence differencing are divided into the following situations: Scenario 1: Perform registration differential between template part 1 and part 1 to be inspected. If G S If -K>0, the initial test fails, but |D S -D M If E is detected, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection is successful, the part to be inspected 1 will be regarded as template part 2. Scenario 2: Perform registration differential between the part to be inspected (2) and the previous qualified part (i.e., template part 2), which is the part to be inspected (1). If G S If -K>0, the initial test fails, but |D S -D M If the result is E, an alarm will be triggered, and a manual re-inspection will be conducted. If the re-inspection fails, the product will be deemed defective. Scenario 3: Perform registration differential between the part to be inspected (3) and the previous qualified part (i.e., template part 2), which is the part to be inspected (1). If G S If -K>0, the initial test fails, and |D S -D M If |≤E, no manual re-inspection is required; the product is directly judged as defective. Scenario 4: Perform registration differential between the part to be inspected (4) and the previous qualified part (i.e., template part 2), which is also the part to be inspected (1). If G S If -K≤0, the initial inspection is qualified, and the part to be inspected 4 will be used as the template part 3 for subsequent inspections; S3. Judge whether the part to be tested is qualified.

2. The visual detection method based on sequence image difference according to claim 1, characterized in that: In S2, the image registration process includes feature region extraction and image sub-pixel subdivision; Line feature extraction is to record the straight-line distance between the surface feature points of the part as D, represent the entire image through a feature center line, and transform the registration of two images from the original registration problem between feature points into the registration of the connection lines between feature points; The image sub-pixel subdivision method is based on the quadratic bilinear interpolation method. Under the existing algorithm framework, the information of unknown points is deduced by means of the information of known points. A pixel is divided into four equal parts in the X and Y directions respectively, that is, the original 1 pixel is equally divided into 16 pixels; the formula is as follows: ； in, Represents physical resolution, unit: mm / pixel; This represents the length of the part being measured in a single direction, in mm. This indicates the image resolution of the part under test in a single direction, in pixels. The number of image interpolations, in seconds.

Images