2.5 D defect detection method and device based on line scanning imaging

Abstract

The invention discloses a 2.5 D defect detection method and device based on line scanning imaging, and the method comprises the steps: collecting a line scanning image which is a to-be-detected workpiece irradiated by a line scanning light source matrix and shot by a line scanning camera, and a light source in the line scanning light source matrix is used for projecting cosine light; splitting the line scanning image into a plurality of phase diagrams based on the arrangement of the line scanning light source matrix; carrying out alignment solution on the phase diagram to obtain a plurality of effect diagrams; according to a preset template drawing, analyzing a feature corresponding relation between the template drawing and the effect drawing, and removing artifacts in the effect drawing to obtain a standard effect drawing; comparing and classifying defect features in the standard effect picture to obtain a defect result, and completing defect detection. Through fusion of arrangement of a line scanning light source matrix and analysis processing of a line scanning image, a solution of a whole process is given, 2.5 D is used for defect detection, the stability is ensured, meanwhile, the method is suitable for a continuous production scene, and the accuracy of industrial defect detection is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of defect detection, and particularly relates to a 2.5D defect detection method and device based on line scanning imaging. BACKGROUND

[0002] In industrial production, defect detection is of great significance, not only as an important means to ensure product quality, but also plays a key role in the entire production process. First, it can directly identify various defects and defects on the production line to ensure high-quality products. Second, detecting product defects early in the production process can prevent the product from entering subsequent processing and assembly stages, thereby reducing material waste and rework costs. In addition, an automated defect detection system can achieve high-speed and high-precision detection, improving production efficiency.

[0003] However, since scratches, pits, and other information appear as small gray level differences in 2D vision and are difficult to distinguish from the background, 2D cameras commonly used in machine vision surface defect detection cannot clearly represent them in images, making it more difficult to improve subsequent detection algorithms.

[0004] Patent application CN117665008A relates to a phase deflection detection system and method for transparent and high-reflective sample surface detection. The system includes a sample stage, a projection light source assembly, a detection camera, and a processing system. The detection method includes the following steps: projecting incident light onto the sample surface through the projection light source assembly, and collecting reflected light of the incident light on the sample through the detection camera; obtaining an encoded fringe pattern of the sample surface and obtaining the positional relationship between the light source, the sample, and the detection camera; calculating the defect features and surface features of the high-reflective sample and the transparent sample surface according to the encoded fringe pattern and the positional relationship between the light source, the sample, and the detection camera. Through one imaging, the three-dimensional point cloud data of the sample can be obtained through system calibration while detecting defects on the sample.

[0005] 2.5D vision illuminates the object surface with cosine light and performs phase calculation on the camera image. Through cosine information, the average, diffuse reflection, specular reflection, reflectivity, and concave-convex shape information of the object surface are obtained. In addition, actual production lines are often continuous production, and 2.5D requires multiple images to be taken. During the movement of the workpiece on the production line, the ordinary area array camera shooting may have pixel misalignment. At the same time, when shooting images with complex textures, the shape map generated by the 2.5D shooting system may have phase mutations caused by color mutations on the object surface. There will be shape artifacts in the shape map at these parts.

[0006] How to use 2.5D for defect detection and improve the accuracy of defect detection in industry is a problem that needs to be solved. SUMMARY

[0007] In response to the defects existing in the above-mentioned prior art, the present invention provides a 2.5D defect detection method and device based on line scan imaging, the method comprising: acquiring a line scan image, wherein the line scan image is a line scan camera photographing a workpiece to be tested illuminated by a line scan light source matrix, and the light source in the line scan light source matrix is ​​used to project cosine light; based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase images; the phase images are aligned and solved to obtain multiple renderings; according to a preset template image, the feature correspondence between the template image and the rendering image is analyzed, the artifacts in the rendering image are removed to obtain a standard rendering image; the defect features in the standard rendering image are compared and classified to obtain defect results, thereby completing defect detection. By integrating the setting of the line scan light source matrix with the analysis and processing of the line scan image, a full-process solution from the setting of the line scan light source matrix to the analysis and processing of the line scan image is provided. Using 2.5D for defect detection ensures stability while being suitable for continuous production scenarios, thereby improving the accuracy of industrial defect detection.

[0008] In a first aspect, the present invention provides a 2.5D defect detection method based on line scan imaging, which specifically includes the following steps: Acquire a line scan image, wherein the line scan image is a shot of a workpiece to be measured by a line scan camera and illuminated by a line scan light source matrix, wherein the light source in the line scan light source matrix is ​​used to project cosine light; Based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase images; Align and solve the phase diagram to obtain multiple effect diagrams; According to the preset template image, the feature correspondence between the template image and the rendering image is analyzed, and the artifacts in the rendering image are removed to obtain the standard rendering image; Compare and classify the defect features in the standard renderings to obtain defect results and complete defect detection.

[0009] Furthermore, the line scanning light source matrix includes n rows of line scanning light sources, each row or column of the line scanning light sources projects a group of light, and the light sources in the line scanning light source matrix are used to project cosine light, which is controlled by the following steps: Based on the optical characteristics of the workpiece to be measured, the brightness of the light source in each row or column is determined separately, where the brightness of the light source is specifically expressed as:

[0010] I i is the brightness of the light source in the i-th row / column, A is the basic brightness, B is the brightness variation range, α is the initial phase of the sinusoidal stripes, is the phase shift; Output light source control signal according to the brightness of the light source.

[0011] Furthermore, based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase maps, specifically including: According to the arrangement of the line-scan light source matrix, the number of light source types projected by the line-scan light source matrix is given; The row number of each row in the line-scan image is taken modulo the number of light source types to obtain a phase map number of each row; According to the phase map number, each row in the line-scan image is reorganized in combination with the row number of each row in the line-scan image to obtain a plurality of phase maps.

[0012] Further, the effect map includes at least one of a mean value map, a diffuse reflection map, a specular map, a gloss ratio map, and a shape map; The phase maps are aligned to obtain a plurality of effect maps, specifically including: According to the phase direction of each phase map, the phase maps are classified to obtain an x-direction phase map set and a y-direction phase map set; Based on the x-direction phase map set, the wrapped phase, the diffuse reflection attribute value, and the specular reflection attribute value in the x direction are analyzed and calculated; Based on the y-direction phase map set, the wrapped phase, the diffuse reflection attribute value, and the specular reflection attribute value in the y direction are analyzed and calculated; The x-direction gradient and the y-direction gradient are obtained by respectively taking the derivative of the wrapped phase in the x direction and the wrapped phase in the y direction; The shape map is obtained by adding the x-direction gradient and the y-direction gradient.

[0013] Further, according to a preset template map, a characteristic correspondence relationship between the template map and the effect map is analyzed to remove artifacts in the effect map to obtain a standard effect map, specifically including: Template features in the template map and effect features in the effect map are respectively extracted; A feature distance matrix is constructed according to the positions of each feature point in the template features and the effect features; The feature distance matrix is analyzed to establish a correspondence relationship between the template features and the effect features; Based on the correspondence relationship between the template features and the effect features, the effect map is aligned with the template map to obtain an aligned effect map; In combination with gradient features in the template map, artifacts in the aligned effect map are filtered to obtain a standard effect map.

[0014] Further, the feature distance matrix is analyzed to establish a correspondence relationship between the template features and the effect features, specifically including: For each feature point in the effect features, the nearest distance value and the second nearest distance value are selected from the feature distance matrix; The distance ratio of the nearest distance value and the second nearest distance value is analyzed; If the distance ratio is within a preset distance range, a corresponding relationship between the feature point in the template features corresponding to the nearest distance value is established; If the distance ratio is not within the preset distance range, the nearest distance value is discarded to be re-determined until the corresponding relationship of all feature points in the effect feature and the template feature is completed.

[0015] Further, the defect features in the standard effect map are compared and classified to obtain a defect result, and the defect detection is completed, specifically including: The standard effect map is binarized to strengthen the defect features, and a first effect map is obtained; The first effect map is closed to fuse the discrete defect feature points, and a second effect map is given; Based on the starting position point of the defect feature region in the second effect map, the minimum circumscribed rectangle of the defect feature region is determined; The aspect ratio of the minimum circumscribed rectangle is analyzed to give the corresponding defect result.

[0016] Further, the first effect map is closed to fuse the discrete defect feature points, and a second effect map is given, specifically including: Based on the pre-set operation unit, the operation unit is traversed in the foreground of the first effect map, if the operation unit includes the foreground, all pixel values in the operation unit are switched to the foreground, the foreground in the first effect map is dilated, and a dilated effect map is obtained; The operation unit is traversed in the foreground of the dilated effect map, if the operation unit includes the background, the center of the operation unit is switched to the background, the foreground of the dilated effect map is eroded, and a second effect map is obtained.

[0017] Further, the control of the light source in the line scanning light source matrix includes the control of the X direction light source and the control of the Y direction light source, wherein the control of each row of light sources in the line scanning light source matrix is the control of the Y direction light source, and the control of each column of light sources in the line scanning light source matrix is the control of the X direction light source: Based on the longitudinal scanning accuracy of the line scanning camera and the moving speed of the workpiece to be measured, the flicker period of the line scanning light source matrix is determined; According to the flicker period, each row or column of light sources in the line scanning light source matrix is controlled.

[0018] In a second aspect, the application also provides a 2.5D defect detection device based on line scanning imaging, which adopts the 2.5D defect detection method based on line scanning imaging as described above, and includes: An image acquisition module is used to acquire a line scanning image, wherein the line scanning image is a line scanning camera shooting a workpiece to be measured irradiated by a line scanning light source matrix, and the light sources in the line scanning light source matrix are used to project cosine light; An image splitting module is used to split the line scanning image into multiple phase images based on the arrangement of the line scanning light source matrix; A phase solving module is configured to align and solve the phase diagram to obtain a plurality of effect diagrams; An artifact removing module is configured to analyze the feature correspondence between the template diagram and the effect diagram according to a preset template diagram, remove the artifacts in the effect diagram, and obtain a standard effect diagram; A defect identifying module is configured to compare and classify the defect features in the standard effect diagram to obtain a defect result and complete the defect detection.

[0019] The 2.5D defect detection method and device based on line scanning imaging provided by the application have at least the following beneficial effects: (1) The line scanning light source matrix is set and controlled to project a plurality of groups of cosine light, a plurality of line scanning images corresponding to a plurality of moments in the movement of the workpiece to be detected are obtained by shooting, and each line scanning image is consistent with the pixel position of the workpiece to be detected, so that the consistency of the pixel position in phase solving is ensured, and at the same time, the analysis and processing of the line scanning image are fused, a full-process solution from the setting and control of the line scanning light source matrix to the analysis and processing of the line scanning image is given, and 2.5D is used for defect detection, which is suitable for the scene of continuous production while ensuring stability, and improves the accuracy of defect detection in industry.

[0020] (2) The line scanning image is split into a conventional phase image, and then the mean value, diffuse reflection, specular reflection, gloss ratio and shape diagram are calculated, so that the dirt information in the background can be filtered out, and scratch detection is facilitated.

[0021] (3) The pixel points are mapped by using the feature point matching of the template diagram and the test diagram, and then the shape diagram is filtered according to the gradient of the template mean value diagram, so that the artifacts are removed, and the accuracy of defect detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flowchart of the 2.5D defect detection method based on line scanning imaging provided by the embodiment of the application is provided. Figure 2 A shooting schematic diagram of the line scanning camera provided by the embodiment of the application is provided. Figure 3 A schematic diagram of the line scanning image provided by the embodiment of the application is provided. Figure 4 A light source arrangement effect diagram in the line scanning light source matrix provided by the embodiment of the application is provided. Figure 5 A phase schematic diagram of the line scanning light source matrix irradiation provided by the embodiment of the application is provided. Figure 6 A flowchart of the split line scanning image provided by the embodiment of the application is provided. Figure 7 A result schematic diagram of the split line scanning image provided by the embodiment of the application is provided. Figure 8 A line-scan 2.5D generated image schematic diagram provided for an embodiment of the present application; Figure 9 A flowchart of determining a standard effect diagram provided for an embodiment of the present application; Figure 10 A flowchart of artifact removal provided for an embodiment of the present application; Figure 11 A flowchart of defect detection provided for an embodiment of the present application; Figure 12 A structural block diagram of a line-scan imaging based 2.5D defect detection device provided for an embodiment of the present application.

[0023] Wherein, 10, a line-scan camera; 20, a workpiece to be measured; 30, a concave-convex defect; 40, a line-scan light source matrix; 201, an image acquisition module; 202, an image splitting module; 203, a phase solving module; 204, an artifact removal module; 205, a defect identification module. DETAILED DESCRIPTION

[0024] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in combination with the drawings of the specification and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0025] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Multiple" generally includes at least two.

[0026] It should also be noted that the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the goods or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such goods or devices. Without more limitations, the element defined by the sentence "including a" does not exclude the presence of other identical elements in the goods or devices including the element.

[0027] With the progress of science and technology, some precision instruments, biomedical and consumer electronics industries, the quality of high-quality optical devices increasingly high demand, to meet more complex imaging function requirements, optical device surface from a simple plane, spherical surface to more and more complex aspheric, free surface. This also puts forward higher requirements for high-precision surface detection of optical lens.

[0028] High light parts of the surface defect detection is an important field of industrial detection, such as automotive interior parts, automotive spraying surface, semiconductor industry wafer and silicon wafer and mirror. The biggest difficulty of this kind of sample detection is the problem of local overexposure.

[0029] In actual projects, a big problem of transparent sample detection is that the upper and lower surfaces of the sample will reflect and refract, and the captured deformation fringe will have superposition problem, so that the correct phase information cannot be extracted for defect detection, resulting in low accuracy of transparent sample defect detection.

[0030] Related patents propose a surface defect detection method and system, the method obtains a 2.5D image set of the target object through a 2.5D imaging system, and the images in the 2.5D image set are background removed to obtain a main 2.5D image set of the target object. The shape map is obtained from the main 2.5D image set, and the surface defect of the target object is detected based on a Gaussian band-pass frequency domain filtering method according to the shape map. The shape map is synthesized by images obtained at different phases of the light source, and characterizes the surface concave-convex situation of the target object, solving the problem of low accuracy of object surface defect detection. The 2.5D image set obtained by the 2.5D imaging system has good imaging effect of subtle defects, and the shape map shows the concave-convex change of the surface through high contrast, so that the defect area is distinguished from the normal surface, improving the accuracy and efficiency of defect detection. However, the use of Gaussian kernel and frequency domain feature calculation shape map feature will make the calculation amount too large, reduce the processing speed, and it does not consider the artifact information caused by color mutation, and can only be applied to objects with less surface texture.

[0031] The conventional area array 2.5D image detection algorithm is generally used for the measured object which is static during shooting, and when applied to objects with complex surface texture, shape artifacts may occur.

[0032] To this end, the application provides a 2.5D defect detection method and device based on line scanning imaging, the method comprising: collecting a line scanning image, wherein the line scanning image is a line scanning camera photographing a workpiece to be measured irradiated by a line scanning light source matrix, and the light sources in the line scanning light source matrix are used for projecting cosine light; based on arrangement of the line scanning light source matrix, the line scanning image is split into a plurality of phase images; the phase images are aligned and solved to obtain a plurality of effect images; according to a preset template image, a feature correspondence relationship between the template image and the effect image is analyzed, false images in the effect image are removed, and a standard effect image is obtained; defect features in the standard effect image are compared and classified to obtain a defect result, and the defect detection is completed. The line scanning light source matrix, false image removal algorithm and image defect detection algorithm are designed according to the actual situation of the production line. In order to be able to photograph the scratch, concave-convex and other information on the surface of a high-reflective and transparent object and ensure the stability during photographing, the method provided in the application adopts the line scanning light source matrix to project a plurality of groups of cosine light at each moment during the movement of the workpiece to be measured when photographing the workpiece to be measured advancing on the production line. In this example, the line scanning light source matrix is provided with 8 rows of line scanning light sources, and for each moment of the workpiece to be measured, 8 rows of corresponding line scanning images can be obtained, and each row of line scanning image is consistent with the pixel position of the workpiece to be measured, so that the consistency of the pixel position during phase solving can be ensured.

[0033] The common shooting mode of the ordinary area array camera 2.5D is to project four groups of cosine light in the horizontal direction and the vertical direction, and it is necessary to ensure that the pixel positions in the eight images are completely consistent. However, the workpiece to be measured on the actual production line is often continuously advancing, and the ordinary area array 2.5D camera cannot ensure the position consistency of the eight images.

[0034] The cosine light refers to light whose intensity distribution changes with the cosine value of the incident angle. Specifically, when light is emitted from a light source to a surface, the relationship between the light intensity I and the incident angle θ (the angle between the light and the normal line of the surface) satisfies the formula: I=I0cosθ, wherein I is the light intensity at the incident angle θ, I0 is the light intensity when the incident angle θ=0°, that is, the light intensity when the light is vertically incident, and cosθ represents the cosine value change of the light intensity with the incident angle. This light intensity distribution law shows that when the incident angle θ increases, the light intensity will gradually decrease, and the decreasing relationship conforms to the law of the cosine function. It can be understood that in the application, the light source brightness of each row or each column in the line scanning light source matrix is a cosine-distributed structured light stripe. The light intensity of the light source is encoded into a cosine function, the deformation of the light after being modulated by the workpiece to be measured is recorded by the line scanning camera, and the cosine (phase) information is used to inversely deduce the surface geometry and reflection properties of the workpiece to be measured.

[0035] 2.5D means introducing depth information on the basis of a two-dimensional image to form an image representation with high information. This representation method is between 2D and 3D, and is usually obtained through monocular vision or depth sensor. Compared with traditional 2D images, 2.5D images increase depth information, that is, each pixel point contains distance information of the point relative to the camera in addition to color information, but since the 2.5D image is usually obtained from a fixed perspective, it cannot provide complete three-dimensional information, such as the depth data of the occluded part.

[0036] As shown in Figure 1 The embodiment of the application provides a 2.5D defect detection method based on line scanning imaging, and the specific steps are as follows: S101: Collecting a line scanning image.

[0037] Specifically, the line scanning image is a line scanning camera shooting a workpiece to be measured irradiated by a line scanning light source matrix, and the light sources in the line scanning light source matrix are used to project cosine light.

[0038] Further, the control of the light sources in the line scanning light source matrix includes X-direction light source control and Y-direction light source control, wherein the control of each row of light sources in the line scanning light source matrix is Y-direction light source control, and the control of each column of light sources in the line scanning light source matrix is X-direction light source control: Based on the longitudinal scanning accuracy of the line scanning camera and the moving speed of the workpiece to be measured, the flashing period of the line scanning light source matrix is determined; According to the flashing period, each row or column of light sources in the line scanning light source matrix is controlled respectively.

[0039] The flashing period is specifically represented as:

[0040] Wherein, T FP is the flashing period of the line scanning light source matrix, CP is the longitudinal scanning accuracy of the line scanning camera, and MS is the moving speed of the workpiece to be measured.

[0041] It can be understood that the flicker period is used to control the accuracy of the line-scan image. In the case where the longitudinal scanning accuracy of the line-scan camera and the moving speed of the workpiece to be measured are determined, the smaller the flicker period, the more information the line-scan image captures in the unit moving distance of the workpiece to be measured, and the more accurate the line-scan image. Conversely, the larger the flicker period, the less information the line-scan image captures in the unit moving distance of the workpiece to be measured, and the lower the accuracy of the line-scan image. When the longitudinal scanning accuracy of the line-scan camera is determined, the shooting range of the line-scan camera is also determined. In combination with the moving speed of the workpiece to be measured, it can be known that the workpiece to be measured will enter the shooting range of the line-scan camera at what time, and the line-scan light source matrix is controlled to project the cosine light to obtain the corresponding line-scan image. The relationship between the flicker period and the longitudinal scanning accuracy of the line-scan camera and the moving speed of the workpiece to be measured can be linear or periodic, which is set according to the actual situation and the accuracy requirement, and is not limited.

[0042] It should be understood that the relationship between the flicker period of the line-scan light source matrix, the longitudinal scanning accuracy of the line-scan camera, and the moving speed of the workpiece to be measured is a mathematical relationship.

[0043] In a specific example, the shooting schematic diagram of the line-scan camera 10 on the production line is shown in Figure 2 The workpiece to be measured 20 moves along the direction of the production line. When the workpiece to be measured 20 enters the shooting range of the line-scan camera 10, the line-scan light source matrix 40 is controlled to project cosine light r1 and r2 of different phases. When the cosine light is projected onto the concave-convex defect 30 of the workpiece to be measured 20, there will be a large phase deviation in the vicinity of the workpiece to be measured 20. The cosine light r1 and r2 are reflected on the concave-convex defect 30 of the workpiece to be measured 20 to obtain r3 and r4 and enter the line-scan camera 10, and the related information of the concave-convex defect 30 on the workpiece to be measured 20 is extracted according to the phase difference Δφ. It can be understood that the line-scan camera 10 and the line-scan light source matrix 40 are controlled in linkage, that is, the line-scan light source matrix 40 projects the cosine light and sends a synchronization signal to the line-scan camera 10 for shooting. The line-scan camera 10 captures the pattern irradiated by the cosine light on the workpiece to be measured 20 to obtain the line-scan image, as shown in Figure 3 .

[0044] In a specific example, the arrangement of the light sources in the line-scan light source matrix is shown in Figure 4As shown, 1 represents the light source combination of each row, the light source combination of each row is Y direction light, 2 is a single light source in the line scanning light source matrix, 3 represents the light source combination of each column, and the light source combination of each column is X direction light. In order to realize phase image shooting in high-speed line scanning state, a line scanning light source matrix capable of high-speed projection of cosine light is proposed. The line scanning light source matrix is composed of LED lamp bead arrangement, and a layer of acrylic plate is laid on the surface during installation, so that the light emitted by the lamp bead is diffused, and the imaging effect is optimized. The light source in the X direction needs to realize that the light source presents a sine wave in the horizontal direction; the X direction is consistent with the row direction of the image when the line scanning camera shoots, and the Y direction is consistent with the column direction of the image.

[0045] The control of the light source in the line scanning light source matrix needs to be controlled in the x direction and the y direction respectively, for example, the 1st, 3rd, 5th and 7th rows are Y direction light, and each row is controlled together during control. The other lamp beads are x direction light, and each column is controlled together, and the light source intensity of each row or each column meets the relationship of the brightness of the cosine light source.

[0046] In a specific example, after the workpiece to be measured moves a specified distance (for example, 0.1 mm), the light source in the line scanning light source matrix irradiates 8 times, the line scanning camera shoots 8 rows, and then moves again and repeats the shooting. And the 8 line scanning images correspond to the imaging results of the line scanning light source matrix irradiating 8 times respectively. As shown, Figure 5 As shown, the 8 images irradiated by the line scanning light source matrix are 4 X direction phase images in the first row and 4 Y direction phase images in the second row.

[0047] In order to be able to correctly project cosine light and realize accurate control of the brightness of each row or each column of light source, a high-performance embedded device is used to send PWM signals to each row and each column of light source, so as to accurately and rapidly control the overall brightness effect.

[0048] Further, the line scanning light source matrix includes n rows of line scanning light sources, each row of line scanning light sources projects a group of light, and is controlled by the following steps: Based on the optical characteristics of the workpiece to be measured, the brightness of each row or each column of light source is determined respectively, wherein the brightness of the light source is specifically represented as:

[0049] I i The brightness of the light source in the i-th row / column, A is the basic brightness, B is the brightness variation range, and a is the initial phase of the sine stripe, is the phase offset; According to the brightness of the light source, a light source control signal is output.

[0050] Taking the x direction light source as an example, the image brightness corresponding to the standard light source brightness is specifically represented as: ; wherein, I is the pixel gray value of the image, A x is the average property of the image, B x is the specular reflection property of the image, a x is the initial phase of the sinusoidal fringe, is the phase shift in the four-step phase shifting method. By respectively controlling the PWM duty cycle of each column or row, the brightness of the light source of each column or row is controlled, so that the corresponding cosine light fringe can be formed after covering the acrylic plate.

[0051] S102: based on the arrangement of the line scanning light source matrix, the line scanning image is split into multiple phase images.

[0052] Further, based on the arrangement of the line scanning light source matrix, the line scanning image is split into multiple phase images, referring to Figure 6 , specifically including: Based on the arrangement of the line scanning light source matrix, the number of light source types projected by the line scanning light source matrix is given; The row number of each row in the line scanning image is taken modulo the number of light source types to obtain the phase image number of each row; According to the phase image number, the rows in the line scanning image are reorganized in combination with the row numbers of the rows in the line scanning image to obtain multiple phase images.

[0053] It should be understood that when the line scanning 2.5D image is captured, the workpiece to be measured on the production line will always move, and the light source will always switch, so that each row of the image is under the imaging effect of a cosine light source. Based on this, the line scanning image is split, and the number of rows is taken modulo the number of light source types to calculate the phase image to which each row belongs.

[0054] In a specific example, the line scanning image has 64 rows, each row is numbered, i.e., the row number is 1-64, and the number of light source types is 8, then the row number of each row is taken modulo, 1 mod 8 = 1, 2 mod 8 = 2, 3 mod 8 = 3, …, 64 mod 8 = 0. The rows with equal remainders are spliced in turn, i.e., the row numbers 1, 9, 17, 25, 33, 41, 49, and 57 are spliced together to obtain a phase image. Similarly, eight phase images can be obtained, as shown in Figure 7 .

[0055] After the line scanning image is split, eight phase images can be obtained. From top to bottom, they are four phases in the x direction and four phases in the y direction.

[0056] S103: aligning and calculating the phase images to obtain multiple effect images.

[0057] The phase images are aligned and calculated to obtain multiple effect images, wherein the effect images include at least one of the average image, the diffuse reflection image, the specular image, the gloss ratio image, and the shape image, specifically including: According to the phase direction of each phase map, the phase maps are classified to obtain an x-direction phase map set and a y-direction phase map set; Based on the x-direction phase map set, the wrapped phase, diffuse reflectance attribute value and specular reflectance attribute value in the x direction are analyzed and calculated; Based on the y-direction phase map set, the wrapped phase, diffuse reflectance attribute value and specular reflectance attribute value in the y direction are analyzed and calculated; The x-direction gradient and the y-direction gradient are obtained by respectively deriving the wrapped phase in the x direction and the wrapped phase in the y direction; The shape map is obtained by adding the x-direction gradient and the y-direction gradient.

[0058] It should be understood that after obtaining the eight phase maps, the wrapped phase, the diffuse reflectance attribute and the specular reflectance attribute in the x direction and the y direction can be calculated by using the phase maps. The wrapped phase is specifically represented as:

[0059] wherein, is the phase of each pixel point, is the phase of the light source projected by the phase map, I i is the gray value of the i-th pixel point, and N is the number of pixel points.

[0060] The mean attribute is the average value of the four phase maps in a single direction, and the mean attribute C of each point is specifically represented as:

[0061] The specular reflectance attribute D is the reflection ability of each point to light, and is specifically represented as:

[0062] After calculating the phase, the mean attribute and the specular reflectance attribute of each pixel point, the mean map, the diffuse reflectance map, the specular map, the gloss ratio map and the shape map can be obtained. The mean map is an image composed of the mean attribute values of each point, the diffuse reflectance map is a spectral map formed by the diffuse reflectance intensity of the material at different wavelengths, the specular map is an image composed of the specular reflectance attribute values of each point, and the gloss ratio map is an image composed of the ratio of the diffuse reflectance attribute value to the specular reflectance attribute value of each point. The shape map is an image composed of the derivative of the wrapped phase, and the phase in the x direction and the y direction is obtained by respectively solving the horizontal and vertical phases and . The x-direction gradient G x is the derivative of the phase in the x direction by using the sobel operator, and is specifically represented as:

[0063] Gradient G in y direction y Derive the phase in y direction for Sobel operator:

[0064] wherein, is Sobel operator in x direction, is Sobel operator in y direction.

[0065] For the problem of phase period change, only need to add recalculate the phase value at the place where the gradient is large.

[0066] By splitting the line-scan image, as Figure 8 shown, the average image, diffuse reflection image, specular image, gloss ratio image, shape image are obtained from top to bottom. The scratch defect is reflected in the shape image, and the gray scale features in the average image, diffuse reflection image, specular image, gloss ratio image are all information of dirt. The dirt information can be clearly seen in the specular image and gloss ratio image.

[0067] By splitting the line-scan image into a conventional phase image, then calculating the average, diffuse reflection, specular reflection, gloss ratio and shape image, the dirt information in the background can be filtered out, which is convenient for scratch detection.

[0068] S104: According to the preset template image, analyze the feature correspondence relationship between the template image and the effect image, remove the artifacts in the effect image, and obtain a standard effect image.

[0069] Further, according to the preset template image, analyze the feature correspondence relationship between the template image and the effect image, remove the artifacts in the effect image, and obtain a standard effect image, refer to Figure 9 , specifically comprising: Respectively extract the template features in the template image and the effect features in the effect image; According to the positions of the feature points in the template features and the effect features, a feature distance matrix is constructed; The feature distance matrix is analyzed to establish the correspondence relationship between the template features and the effect features; Based on the correspondence relationship between the template features and the effect features, the effect image is aligned with the template image to obtain an aligned effect image; Combined with the gradient features in the template image, the artifacts in the aligned effect image are filtered to obtain a standard effect image.

[0070] It can be understood that when detecting a workpiece with complex surface information, the phase may suddenly change at the place of brightness change, which may bring artifacts to the shape image. The process of removing artifacts is refer to Figure 10Firstly, the good product image is taken as a template image, then the mean image of the template image is calculated, the feature points of the template image are extracted, and the gradient information of the template image is calculated; then based on the mean image, shape image and other information of the effect image, the feature point information in the effect image is extracted to obtain the effect feature, and the effect feature is matched with the template feature of the template image. Then the shape image in the effect image is subjected to affine transformation and mapped into the template image, and the corresponding image of the effect image is divided by the gradient at the corresponding position of the template image, so that the artifact information caused by color change can be filtered out to obtain a standard effect image. The good product image is an image that meets specific quality standards and requirements.

[0071] Further, the feature distance matrix is analyzed to establish the correspondence relationship between the template feature and the effect feature, specifically including: For each feature point in the effect feature, the nearest distance value and the second nearest distance value are selected from the feature distance matrix; The distance ratio of the nearest distance value and the second nearest distance value is analyzed; If the distance ratio is within the preset distance range, the feature point in the template feature corresponding to the nearest distance value is established to have a correspondence relationship; If the distance ratio is not within the preset distance range, the nearest distance value is discarded and re-determined until the correspondence relationship between all feature points in the effect feature and the template feature is completed.

[0072] In a specific embodiment, the Euclidean distance between each feature point position in the effect feature and each feature point position in the template feature is calculated, and each element in the feature distance matrix is constructed as the Euclidean distance between the i-th feature point in the effect feature and the j-th feature point in the template feature. Each feature point in the effect feature is analyzed in turn to obtain the nearest distance value and the second nearest distance value corresponding to the feature point, and the ratio of the nearest distance value and the second nearest distance value, i.e., the distance ratio, is calculated. If the distance ratio is smaller, it means that the difference between the nearest distance value and the second nearest distance value is larger, and the feature points corresponding to the nearest distance value and the second nearest distance value have higher distinguishability, so the feature point corresponding to the nearest distance value is more reliable and a correspondence relationship can be established. Conversely, if the distance ratio is larger, it means that the difference between the nearest distance value and the second nearest distance value is smaller, and the feature points corresponding to the nearest distance value and the second nearest distance value have lower distinguishability, so the feature point corresponding to the nearest distance value is less reliable and a correspondence relationship cannot be established. At this time, the feature point corresponding to the nearest distance value is discarded and a new nearest distance value is re-determined until the distance ratio is within the preset distance range and a correspondence relationship can be established.

[0073] Based on the correspondence between the template features and the effect features, the effect picture is subjected to affine transformation and aligned with the template picture to obtain an aligned effect picture. After obtaining the aligned effect picture, the gradient features of each feature point in the template picture are calculated, and the feature points in the effect picture are divided by the gradient features at the corresponding positions in the template picture, so that the artifacts in the aligned effect picture can be filtered to obtain a standard effect picture.

[0074] The gradient feature of the image is a two-dimensional vector representing the rate of change of image intensity at each pixel position. For a gray-scale image I(x, y), the gradient feature At position (x, y) can be represented as:

[0075] wherein, is the partial derivative of I(x, y) in the horizontal direction, is the partial derivative of I(x, y) in the vertical direction.

[0076] The gradient feature can be calculated by Sobel operator, Prewitt operator, Roberts operator, etc. In this example, Sobel operator is used for calculation. In a specific example, for the pixel value of any feature point in the template picture, the convolution kernel of the Sobel operator in the x direction and the convolution kernel in the y direction are respectively convolved to obtain the x gradient component and the y gradient component. The square root of the sum of the x gradient component and the y gradient component can obtain the gradient feature corresponding to the feature point. Similarly, the gradient features of all feature points in the template picture can be calculated. By traversing each feature point in the aligned effect picture, the pixel value of each feature point is divided by the gradient feature of the corresponding feature point in the template picture to remove the artifacts in the aligned effect picture and obtain a standard effect picture.

[0077] For the problem of 2.5D shape artifacts caused by dramatic color changes, the feature points of the template picture and the test picture are matched to map the pixel points, and then the shape picture is filtered according to the gradient of the template mean picture to remove the artifacts and improve the accuracy of defect detection.

[0078] S105: Comparing and classifying the defect features in the standard effect picture to obtain a defect result and complete defect detection.

[0079] Further, the defect features in the standard effect picture are compared and classified to obtain a defect result and complete defect detection, as described in Figure 11 , specifically including: The standard effect picture is binarized to strengthen the defect features and obtain a first effect picture; The first effect picture is subjected to a closing operation to fuse discrete defect feature points and give a second effect picture; Based on the starting position point of the defect feature region in the second effect map, a minimum circumscribed rectangle of the defect feature region is determined; The aspect ratio of the minimum circumscribed rectangle is analyzed to give a corresponding defect result.

[0080] Further, a closing operation is performed on the first effect map to fuse the discrete defect feature points and give a second effect map, specifically including: Based on a pre-set operation unit, the operation unit is traversed in the foreground of the first effect map, if the operation unit includes the foreground, all pixel values in the operation unit are switched to the foreground, the foreground in the first effect map is dilated to obtain a dilated effect map; The operation unit is traversed in the foreground of the dilated effect map, if the operation unit includes the background, the center of the operation unit is switched to the background, the foreground of the dilated effect map is eroded to obtain the second effect map.

[0081] It can be understood that after the standard effect map is calculated, a binarization operation is performed on the standard effect map, and at this time, due to problems such as shooting noise and angle, a defect may be discontinuous, therefore, a closing operation is performed on the standard effect map to connect the defect parts together and give a minimum circumscribed rectangle of the defect, show the detected defects in the line scan image, and obtain a defect result.

[0082] It can be understood that after the standard effect map is calculated, a binarization operation is performed on the standard effect map, and at this time, due to problems such as shooting noise and angle, a defect may be discontinuous, therefore, a closing operation is performed on the standard effect map to connect the defect parts together and give a minimum circumscribed rectangle of the defect, show the detected defects in the line scan image, and obtain a defect result.

[0083] The preset operation unit is a pixel block composed of multiple pixels, which can be a circle with a radius r, a cross with a length L, or a rectangle, and is not limited in this regard. Each operation unit has a center, and the center of the circle is the center, and the center of the cross is the intersection point of the horizontal and vertical directions. In a specific embodiment, the center of the operation unit is traversed in the foreground of the first effect map, and if the operation unit includes the foreground, all pixel values in the operation unit are switched to the foreground, and the foreground in the first effect map is expanded to obtain an expansion effect map. For example, the operation unit has 5 pixel values, and in the traversal process, the center of the operation unit sequentially passes through each pixel in the foreground, and the background pixel values covered by other operation units are switched to the foreground. After the traversal is completed, the expansion effect map is obtained. Then, the center of the operation unit is traversed in the foreground of the expansion effect map, and if the operation unit includes the background, the center of the operation unit is switched to the background, and the foreground of the expansion effect map is eroded to obtain a second effect map. For example, in the traversal process, if one of the 5 pixel values of the operation unit is background, the center of the operation unit is switched to background, and after the traversal is completed, the second effect map is obtained.

[0084] After obtaining the second effect map, a starting position point is determined from the foreground, which can be the position point with the smallest horizontal and vertical coordinates, the position point with the largest horizontal and vertical coordinates, or other edge points of the foreground. With the starting position point as an end point, the horizontal direction is taken as the starting direction for rotation, and points tangent to the foreground during rotation are recorded. For each tangent point, an inscribed rectangle is constructed, the areas of the inscribed rectangles are analyzed, and the inscribed rectangle with the smallest area is determined as the minimum inscribed rectangle. The aspect ratio of the minimum inscribed rectangle is analyzed, and the defect features are classified according to the aspect ratio. Defects with an aspect ratio close to 1 are convex or concave features, and defects with an aspect ratio reaching a preset scratch threshold are scratch information, and the corresponding defect results are obtained.

[0085] The 2.5D defect detection method based on line scanning imaging provided by the present application gives a full-process solution from the setting of the line scanning light source matrix to the analysis and processing of the line scanning image, and can be applied to a continuous production scene. Compared with the traditional 2.5D detection, the present application has a wider application range, better stability, and lower light source cost.

[0086] Referring to Figure 12 The embodiment of the present application provides a 2.5D defect detection device based on line scanning imaging, which comprises: An image acquisition module 201 is configured to acquire a line scanning image, wherein the line scanning image is a line scanning camera photographing a workpiece to be measured irradiated by a line scanning light source matrix, and the light sources in the line scanning light source matrix are configured to project cosine light; An image splitting module 202 is configured to split the line scanning image into multiple phase images based on the arrangement of the line scanning light source matrix. The phase solving module 203 is configured to align and solve the phase diagram to obtain a plurality of effect diagrams; The artifact removing module 204 is configured to analyze the feature correspondence between the template diagram and the effect diagram according to the preset template diagram, remove the artifact in the effect diagram, and obtain a standard effect diagram. The defect identifying module 205 is configured to compare and classify the defect features in the standard effect diagram to obtain a defect result, and complete the defect detection.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described modules can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0088] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to these embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A 2.5D defect detection method based on line scan imaging, characterized in that, The method comprises the following steps: Collecting a line scanning image, wherein the line scanning image is taken by a line scanning camera and irradiates a workpiece to be measured by a line scanning light source matrix, and the light sources in the line scanning light source matrix are used for projecting cosine light; Based on the arrangement of the line scanning light source matrix, the line scanning image is divided into a plurality of phase images; The phase images are aligned and solved to obtain a plurality of effect images; According to a preset template image, the feature correspondence relationship between the template image and the effect image is analyzed, the artifacts in the effect image are removed, and a standard effect image is obtained; The defect features in the standard effect image are compared and classified to obtain a defect result, and the defect detection is completed.

2. The 2.5D defect detection method based on line scan imaging according to claim 1, characterized in that: The line scanning light source matrix comprises n rows of line scanning light sources, each row or column of line scanning light sources projects a group of light, and the light sources in the line scanning light source matrix are used for projecting cosine light, and the projection of the cosine light is controlled by the following steps: Based on the optical characteristics of the workpiece to be measured, the brightness of each row or column of light sources is determined respectively, wherein the brightness of the light source is specifically represented as: ； I i L(i) = A + B sin(2πi / N + α) for the i-th row / column of light sources, A is the base luminance, B is the luminance variation range, and α is the initial phase of the sinusoidal stripe, is the phase offset; According to the light source brightness, the light source control signal is output.

3. The line-scan based 2.5D defect detection method of claim 1, wherein, Based on the arrangement of the line scanning light source matrix, the line scanning image is divided into a plurality of phase images, specifically including: Based on the arrangement of the line scanning light source matrix, the number of light source types projected by the line scanning light source matrix is given; The row number of each row in the line scanning image is taken modulo the number of light source types to obtain the phase image number of each row; According to the phase image number, the rows in the line scanning image are reorganized according to the row number of each row in the line scanning image to obtain a plurality of phase images.

4. The line-scan based 2.5D defect detection method of claim 1 or 3, wherein, The effect image includes at least one of the mean value image, the diffuse reflection image, the specular image, the gloss ratio image and the shape image; The phase images are aligned and solved to obtain a plurality of effect images, specifically including: According to the phase direction of each phase image, the phase images are classified to obtain an x-direction phase image set and a y-direction phase image set; Based on the x-direction phase image set, the wrapped phase, the diffuse reflection attribute value and the specular reflection attribute value in the x-direction are analyzed and calculated; Based on the y-direction phase image set, the wrapped phase, the diffuse reflection attribute value and the specular reflection attribute value in the y-direction are analyzed and calculated; The wrapped phase in the x-direction and the wrapped phase in the y-direction are differentiated respectively to obtain the x-direction gradient and the y-direction gradient; The x-direction gradient and the y-direction gradient are added to obtain the shape image.

5. The line-scan based 2.5D defect detection method of claim 1, wherein, According to the preset template image, the feature correspondence relationship between the template image and the effect image is analyzed, the artifacts in the effect image are removed, and a standard effect image is obtained, specifically including: Template features in the template image and effect features in the effect image are extracted respectively; According to the positions of the feature points in the template features and the effect features, a feature distance matrix is constructed; The feature distance matrix is analyzed to establish the correspondence relationship between the template features and the effect features; Based on the correspondence relationship between the template features and the effect features, the effect image is aligned with the template image to obtain an aligned effect image; Combined with the gradient features in the template image, the artifacts in the aligned effect image are filtered to obtain a standard effect image.

6. The line-scan based 2.5D defect detection method of claim 5, wherein, The feature distance matrix is analyzed to establish the correspondence relationship between the template features and the effect features, specifically including: For each feature point in the effect features, the nearest distance value and the second nearest distance value are selected from the feature distance matrix; The distance ratio of the nearest distance value and the second nearest distance value is analyzed. If the distance ratio is within the preset distance range, the feature points in the template feature corresponding to the nearest distance value are established in correspondence; If the distance ratio is not within the preset distance range, the nearest distance value is discarded and re-determined until the correspondence of all feature points in the effect feature and the template feature is completed.

7. The line-scan based 2.5D defect detection method of claim 1, wherein, The defect features in the standard effect map are compared and classified to obtain a defect result, and defect detection is completed, specifically including: The standard effect map is binarized to strengthen the defect features and obtain a first effect map; A closed operation is performed on the first effect map to fuse discrete defect feature points and give a second effect map; Based on the starting position point of the defect feature region in the second effect map, the minimum circumscribed rectangle of the defect feature region is determined; The aspect ratio of the minimum circumscribed rectangle is analyzed to give the corresponding defect result.

8. The line-scan based 2.5D defect detection method of claim 7, wherein, A closed operation is performed on the first effect map to fuse discrete defect feature points and give a second effect map, specifically including: Based on a pre-set operation unit, the operation unit is traversed in the foreground of the first effect map, if the operation unit includes the foreground, all pixel values in the operation unit are switched to the foreground, the foreground in the first effect map is dilated to obtain a dilated effect map; The operation unit is traversed in the foreground of the dilated effect map, if the operation unit includes the background, the center of the operation unit is switched to the background, the foreground of the dilated effect map is eroded to obtain a second effect map.

9. The line-scan based 2.5D defect detection method of claim 1, wherein, The control of the light sources in the line-scan light source matrix includes X-direction light source control and Y-direction light source control, wherein the control of each row of light sources in the line-scan light source matrix is Y-direction light source control, and the control of each column of light sources in the line-scan light source matrix is X-direction light source control: Based on the longitudinal scanning accuracy of the line-scan camera and the moving speed of the workpiece to be measured, the flicker period of the line-scan light source matrix is determined; According to the flicker period, each row or column of light sources in the line-scan light source matrix is controlled respectively.

10. A line-scan based 2.5D defect detection apparatus, characterized by, The 2.5D defect detection method based on line-scan imaging as claimed in any one of claims 1-9, comprising: An image acquisition module for acquiring a line-scan image, wherein the line-scan image is a line-scan camera photographing a workpiece to be measured irradiated by a line-scan light source matrix, and the light sources in the line-scan light source matrix are used for projecting cosine light; An image splitting module for splitting the line-scan image into multiple phase maps based on the arrangement of the line-scan light source matrix; A phase calculation module for aligning and calculating the phase maps to obtain multiple effect maps; An artifact removal module for analyzing the feature correspondence between a template map and an effect map according to a pre-set template map, removing artifacts in the effect map, and obtaining a standard effect map; A defect identification module for comparing and classifying defect features in the standard effect map to obtain a defect result and complete defect detection.

Images