A 2.5D defect detection method and apparatus based on line scan imaging

Abstract

This invention discloses a 2.5D defect detection method and apparatus based on line scan imaging. The method includes: acquiring a line scan image, wherein the line scan image is a photograph taken by a line scan camera of the workpiece under test illuminated by a line scan light source matrix, and the light sources in the line scan light source matrix are used to project cosine light; based on the arrangement of the line scan light source matrix, the line scan image is divided into multiple phase images; the phase images are aligned and solved to obtain multiple effect images; according to a preset template image, the feature correspondence between the template image and the effect images is analyzed, and artifacts in the effect images are removed to obtain a standard effect image; the defect features in the standard effect images are compared and classified to obtain the defect result, thus completing the defect detection. By integrating the setting of the line scan light source matrix and the analysis and processing of the line scan image, a complete solution is provided. Using 2.5D for defect detection, while ensuring stability, it is suitable for continuous production scenarios and improves the accuracy of defect detection in industry.

Description

Technical Field

[0001] This invention belongs to the technical field of defect detection, specifically relating to a 2.5D defect detection method and device based on line scanning imaging. Background Technology

[0002] In industrial production, defect detection is of great significance, not only as an important means of ensuring product quality but also as a key component of the entire production process. Firstly, it can directly identify various flaws and defects on the production line, ensuring high product quality. Secondly, detecting product defects early in production can prevent them from entering subsequent processing and assembly stages, thereby reducing material waste and rework costs. Furthermore, automated defect detection systems can achieve high-speed, high-precision detection, improving production efficiency.

[0003] However, since scratches, dents, and shallow dirt appear as small grayscale differences in 2D vision and are difficult to distinguish from the background, 2D cameras commonly used for machine vision surface defect detection cannot clearly show them in the image, which increases the difficulty of subsequent detection algorithms.

[0004] Patent application CN117665008A relates to a phase deflection detection system and method for detecting the surfaces of transparent and highly reflective samples. The system includes a sample stage, a projection light source assembly, a detection camera, and a processing system. The detection method includes the following steps: projecting incident light onto the sample surface through the projection light source assembly, and collecting the reflected light on the sample through the detection camera; obtaining a coded fringe pattern of the sample surface and acquiring the positional relationship between the light source, the sample, and the detection camera; and calculating the defect features and surface features of highly reflective and transparent samples based on the coded fringe pattern and the positional relationship between the light source, the sample, and the detection camera. Through a single imaging process, defect detection of the sample can be performed simultaneously, and the system calibration can also obtain the sample's three-dimensional point cloud data.

[0005] 2.5D vision illuminates an object's surface with cosine light and performs phase calculations on the camera images. The cosine information is used to obtain the object's mean, diffuse, specular, reflectivity, and surface texture information. However, actual production lines are often continuous, and 2.5D requires capturing multiple images. During workpiece movement on the production line, ordinary area scan cameras can experience pixel misalignment. Furthermore, when capturing images with complex textures, the shape map generated by the 2.5D imaging system may contain phase abrupt changes caused by abrupt color transitions on the object's surface, resulting in shape artifacts in these areas.

[0006] How to use 2.5D for defect detection and improve the accuracy of defect detection in industry is a problem that needs to be solved. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a 2.5D defect detection method and apparatus based on line scan imaging. The method includes: acquiring a line scan image, wherein the line scan image is captured by a line scan camera of the workpiece under test illuminated by a line scan light source matrix, and the light sources in the line scan light source matrix are used to project cosine light; based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase images; the phase images are aligned and solved to obtain multiple effect images; according to a preset template image, the feature correspondence between the template image and the effect images is analyzed, and artifacts in the effect images are removed to obtain a standard effect image; the defect features in the standard effect images are compared and classified to obtain the defect result, thus completing the defect detection. By integrating the setting of the line scan light source matrix and the analysis and processing of the line scan image, a complete solution from setting the line scan light source matrix to analyzing and processing the line scan image is provided. Using 2.5D for defect detection, while ensuring stability, it is suitable for continuous production scenarios and improves the accuracy of defect detection in industry.

[0008] In a first aspect, the present invention provides a 2.5D defect detection method based on line scan imaging, specifically including the following steps:

[0009] Acquire line scan images, where the line scan image is the workpiece to be measured captured by the line scan camera and illuminated by the line scan light source matrix. The light source in the line scan light source matrix is ​​used to project cosine light.

[0010] Based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase maps;

[0011] The phase map is aligned and solved to obtain multiple effect maps;

[0012] Based on the preset template image, analyze the feature correspondence between the template image and the rendering image, remove the artifacts in the rendering image, and obtain the standard rendering image;

[0013] The defect features in the standard rendering are compared and classified to obtain the defect results, thus completing the defect detection.

[0014] Furthermore, the line-scan light source matrix includes n rows of line-scan light sources, with each row or column of line-scan light sources projecting a set of light. The light sources in the line-scan light source matrix are used to project cosine light, and are controlled through the following steps:

[0015] Based on the optical characteristics of the workpiece under test, the brightness of the light source for each row or column is determined, where the light source brightness is specifically expressed as:

[0016]

[0017] I i Let A be the light source brightness in the i-th row / column, B be the base brightness, and α be the brightness variation range. α is the initial phase of the sinusoidal fringes. For phase shift;

[0018] Based on the brightness of the light source, output a light source control signal.

[0019] Furthermore, based on the arrangement of the line scan light source matrix, the line scan image is divided into multiple phase maps, specifically including:

[0020] Based on the arrangement of the line scan light source matrix, the number of light source types projected by the line scan light source matrix is ​​given;

[0021] The row number of each row in the line scan image is moduloed by the number of light source types to obtain the phase map number of each row.

[0022] Based on the phase map number and the row number of each row in the line scan image, the rows in the line scan image are recombined to obtain multiple phase maps.

[0023] Furthermore, the renderings include at least one of the following: mean plot, diffuse plot, specular plot, gloss ratio plot, and shape plot;

[0024] The phase map is aligned and solved to obtain multiple result images, including:

[0025] Based on the phase direction of each phase map, the phase maps are classified to obtain the x-direction phase map set and the y-direction phase map set;

[0026] Based on the x-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the x-direction are analyzed and calculated.

[0027] Based on the y-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the y-direction are analyzed and calculated.

[0028] Differentiate the wrapping phase in the x-direction and the wrapping phase in the y-direction respectively to obtain the gradient in the x-direction and the gradient in the y-direction;

[0029] Add the gradients in the x and y directions to obtain the shape diagram.

[0030] Furthermore, based on the preset template image, the correspondence between the features of the template image and the rendered image is analyzed, and artifacts in the rendered image are removed to obtain the standard rendered image, specifically including:

[0031] Extract the template features from the template image and the effect features from the effect image respectively;

[0032] Construct a feature distance matrix based on the positions of each feature point in the template features and effect features;

[0033] Analyze the feature distance matrix to establish the correspondence between template features and effect features;

[0034] Based on the correspondence between template features and effect features, the effect image is aligned with the template image to obtain the aligned effect image;

[0035] By combining the gradient features in the template image, the artifacts in the alignment effect image are filtered out to obtain the standard effect image.

[0036] Furthermore, the feature distance matrix is ​​analyzed to establish the correspondence between template features and effect features, specifically including:

[0037] For each feature point in the effect features, the nearest and second nearest distance values ​​are selected from the feature distance matrix;

[0038] Analyze the distance ratio between the nearest and second-nearest distance values;

[0039] If the distance ratio is within the preset distance range, a correspondence is established with the feature point in the template feature corresponding to the nearest distance value;

[0040] If the distance ratio is not within the preset distance range, discard the nearest distance value and re-determine it until the correspondence between all feature points in the effect feature and the template feature is completed.

[0041] Furthermore, the defect features in the standard rendering are compared and classified to obtain defect results, thus completing defect detection. This specifically includes:

[0042] The standard rendering is binarized to enhance defect features, resulting in the first rendering.

[0043] A closing operation is performed on the first effect image to fuse the discrete defect feature points and give the second effect image;

[0044] Based on the starting point of the defect feature region in the second rendering, determine the minimum bounding rectangle of the defect feature region;

[0045] Analyze the aspect ratio of the minimum bounding rectangle and provide the corresponding defect results.

[0046] Furthermore, a closing operation is performed on the first effect image to fuse the discrete defect feature points, resulting in a second effect image, specifically including:

[0047] Based on a pre-defined unit of operation, the unit of operation is traversed in the foreground of the first effect image. If the unit of operation includes the foreground, all pixel values ​​in the unit of operation are switched to the foreground, and the foreground in the first effect image is expanded to obtain an expanded effect image.

[0048] The operation unit is traversed in the foreground of the expansion effect image. If the operation unit includes the background, the center of the operation unit is switched to the background, and the foreground of the expansion effect image is eroded to obtain the second effect image.

[0049] Furthermore, the control of the light sources in the line scan light source matrix includes the control of the light sources in the X direction and the control of the light sources in the Y direction. Specifically, the control of the light sources in each row of the line scan light source matrix is ​​the control of the light sources in the Y direction, and the control of the light sources in each column of the line scan light source matrix is ​​the control of the light sources in the X direction.

[0050] The scintillation period of the line scan light source matrix is ​​determined based on the longitudinal scanning accuracy of the line scan camera and the moving speed of the workpiece under test.

[0051] Based on the flicker period, the light sources in each row or column of the line scan light source matrix are controlled separately.

[0052] Secondly, the present invention also provides a 2.5D defect detection device based on line scanning imaging, employing any one of the above-mentioned 2.5D defect detection methods based on line scanning imaging, including:

[0053] The image acquisition module is used to acquire line scan images, wherein the line scan image is the workpiece under test captured by the line scan camera and illuminated by the line scan light source matrix, and the light source in the line scan light source matrix is ​​used to project cosine light;

[0054] The image splitting module is used to split the line scan image into multiple phase maps based on the arrangement of the line scan light source matrix;

[0055] The phase resolution module is used to align and resolve the phase map to obtain multiple effect maps;

[0056] The artifact removal module is used to analyze the feature correspondence between the template image and the effect image based on the preset template image, remove artifacts in the effect image, and obtain the standard effect image.

[0057] The defect identification module is used to compare and classify the defect features in the standard rendering to obtain the defect results and complete the defect detection.

[0058] The present invention provides a 2.5D defect detection method and apparatus based on line scanning imaging, which has at least the following beneficial effects:

[0059] (1) By setting and controlling the line scan light source matrix, multiple sets of cosine light are projected by the line scan light source matrix. Multiple images are captured at multiple moments during the movement of the workpiece to be tested, and the pixel positions of each line scan image are consistent with those of the workpiece to be tested, ensuring the consistency of pixel positions during phase calculation. At the same time, the analysis and processing of the line scan images are integrated, and a complete solution from setting and controlling the line scan light source matrix to analyzing and processing the line scan images is given. 2.5D is used for defect detection. While ensuring stability, it is suitable for continuous production scenarios and improves the accuracy of defect detection in industry.

[0060] (2) By splitting the line scan image into a regular phase image, and then calculating the mean, diffuse reflection, specular reflection, gloss ratio and shape map, dirt information in the background can be filtered out, which facilitates scratch detection.

[0061] (3) By using feature point matching between the template image and the test image, the pixels are mapped, and the shape image is filtered according to the gradient of the template mean image, the artifacts are eliminated, and the accuracy of defect detection is improved. Attached Figure Description

[0062] Figure 1 A flowchart of a 2.5D defect detection method based on line scan imaging provided in an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of a line scan camera provided in an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a line scan image provided in an embodiment of the present invention; Figure 4 This is a diagram illustrating the arrangement of light sources in a line scan light source matrix provided in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of the phase of a line-scanned light source matrix illumination provided in an embodiment of the present invention;

[0066] Figure 6 A flowchart of a split line scan image provided in an embodiment of the present invention;

[0067] Figure 7 This is a schematic diagram of the result of a split line scan image provided in an embodiment of the present invention;

[0068] Figure 8 This is a schematic diagram of a 2.5D image generated by line scanning according to an embodiment of the present invention;

[0069] Figure 9 A flowchart for determining the standard effect diagram provided in an embodiment of the present invention;

[0070] Figure 10 A flowchart for artifact removal provided in an embodiment of the present invention;

[0071] Figure 11 A flowchart for defect detection provided in an embodiment of the present invention;

[0072] Figure 12 The structural block diagram of the 2.5D defect detection device based on line scanning imaging provided in the embodiments of the present invention is shown.

[0073] Among them, 10 is the line scan camera; 20 is the workpiece to be tested; 30 is the concave and convex defects; 40 is the line scan light source matrix; 201 is the image acquisition module; 202 is the image segmentation module; 203 is the phase calculation module; 204 is the artifact removal module; and 205 is the defect recognition module. Detailed Implementation

[0074] To better understand the above technical solutions, a detailed description of the solutions will be provided below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0075] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0076] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0077] With the advancement of technology, industries such as precision instruments, biomedicine, and consumer electronics are placing increasingly higher demands on the quality of optical components. To meet the needs of more complex imaging functions, the surface shapes of optical components are evolving from simple planar and spherical surfaces to increasingly complex non-spherical and freeform surfaces. This also places high demands on the high-precision surface shape inspection of optical lenses.

[0078] Surface defect inspection of highly reflective components is an important area of ​​industrial inspection, including automotive interior parts, automotive painted surfaces, semiconductor wafers and silicon wafers, and mirrors. The biggest challenge in inspecting these samples is dealing with localized overexposure.

[0079] In practical projects, a major problem in the inspection of transparent samples is that reflection and refraction occur on the upper and lower surfaces of the sample, and the captured deformed fringes may overlap, making it impossible to extract the correct phase information for defect detection, resulting in low accuracy of defect detection for transparent samples.

[0080] A related patent proposes a surface defect detection method and system. The method acquires a 2.5D image set of the target object using a 2.5D imaging system. Background removal is performed on the images in the 2.5D image set to obtain a 2.5D image set of the main body of the target object. A shape map is then extracted from this main body 2.5D image set. Based on a Gaussian bandpass frequency domain filtering method, surface defect detection is performed on the target object according to the shape map. The shape map is synthesized from images acquired at different phases of the light source, characterizing the surface unevenness of the target object. This solves the problem of low accuracy in surface defect detection. The 2.5D image set acquired by the 2.5D imaging system provides good imaging of subtle defects, and the shape map, by displaying the surface unevenness with high contrast, distinguishes defect areas from normal surfaces, improving the accuracy and efficiency of defect detection. However, the method uses Gaussian kernels and frequency domain features to calculate shape map features, resulting in excessive computation and reduced processing speed. Furthermore, it does not consider artifact information caused by color abrupt changes, limiting its applicability to objects with minimal surface texture.

[0081] Currently, conventional 2.5D image detection algorithms are generally used for objects that are stationary during shooting, and shape artifacts may occur when applied to objects with complex surface textures.

[0082] To address this, the present invention provides a 2.5D defect detection method and apparatus based on line scan imaging. The method includes: acquiring a line scan image, wherein the line scan image is a photograph taken by a line scan camera of the workpiece to be tested illuminated by a line scan light source matrix, and the light sources in the line scan light source matrix are used to project cosine light; based on the arrangement of the line scan light source matrix, the line scan image is divided into multiple phase images; the phase images are aligned and solved to obtain multiple effect images; according to a preset template image, the feature correspondence between the template image and the effect images is analyzed, and artifacts in the effect images are removed to obtain a standard effect image; the defect features in the standard effect images are compared and classified to obtain the defect result, thus completing the defect detection. A line scan light source matrix, an artifact removal algorithm, and an image defect detection algorithm are designed specifically for the actual conditions of a production line. In order to capture information such as scratches and bumps on the surface of highly reflective and transparent objects and ensure the stability of the shooting process, the method provided in this application projects multiple sets of cosine light using a line scan light source matrix at each moment of the workpiece moving on the production line. In this example, the line scan light source matrix is ​​set with 8 rows of line scan light sources. For each moment of the workpiece, the corresponding 8 rows of line scan images can be obtained, and the pixel position of each row of line scan images is consistent with that of the workpiece. This ensures the consistency of pixel position during phase calculation.

[0083] The common shooting method of ordinary 2.5D area array cameras is to project four sets of cosine light in the horizontal and four sets in the vertical directions. It is necessary to ensure that the pixel positions in the eight images are completely consistent. However, the workpieces to be tested on the actual production line are often moving continuously, and ordinary 2.5D area array cameras cannot guarantee that the positions of the eight images are consistent.

[0084] Cosine light refers to light whose intensity distribution varies with the cosine value of the incident angle. Specifically, when light rays strike a surface from a light source, the relationship between the light intensity I and the incident angle θ (the angle between the light ray and the surface normal) satisfies the formula: I = I0cosθ, where I is the light intensity at the incident angle θ, I0 is the light intensity when the incident angle θ = 0°, i.e., when the light ray is incident perpendicularly, and cosθ represents the change in light intensity with the cosine value of the incident angle. This light intensity distribution law indicates that as the incident angle θ increases, the light intensity gradually decreases, and this decreasing relationship conforms to the law of the cosine function. It can be understood that in this invention, the light source brightness of each row or column in the line scan light source matrix is ​​a structured light stripe with a cosine distribution in space. The illuminance of the light source is encoded as a cosine function, and its deformation after being modulated by the workpiece under test is recorded by a line scan camera. Then, the geometric and reflection properties of the workpiece surface are deduced using the cosine (phase) information.

[0085] 2.5D refers to the introduction of depth information into a two-dimensional image, forming an image representation with height information. This representation method lies between 2D and 3D and is typically acquired through monocular vision or a depth sensor. Compared to traditional 2D images, 2.5D images add depth information; that is, each pixel contains not only color information but also distance information relative to the camera. However, since 2.5D images are usually acquired from a fixed viewpoint, they cannot provide complete three-dimensional information, such as depth data for occluded areas.

[0086] like Figure 1 As shown, this embodiment of the invention provides a 2.5D defect detection method based on line scan imaging, and the specific steps are as follows:

[0087] S101: Acquire line scan images.

[0088] Specifically, the line scan image is a photograph taken by a line scan camera of the workpiece under test illuminated by a line scan light source matrix, and the light source in the line scan light source matrix is ​​used to project cosine light.

[0089] Furthermore, the control of the light sources in the line scan light source matrix includes the control of the light sources in the X direction and the control of the light sources in the Y direction. Specifically, the control of the light sources in each row of the line scan light source matrix is ​​the control of the light sources in the Y direction, and the control of the light sources in each column of the line scan light source matrix is ​​the control of the light sources in the X direction.

[0090] The scintillation period of the line scan light source matrix is ​​determined based on the longitudinal scanning accuracy of the line scan camera and the moving speed of the workpiece under test.

[0091] Based on the flicker period, the light sources in each row or column of the line scan light source matrix are controlled separately.

[0092] The flashing period is specifically expressed as:

[0093]

[0094] Among them, T FP denoted as the scintillation period of the line scan light source matrix, CP as the longitudinal scanning accuracy of the line scan camera, and MS as the moving speed of the workpiece under test.

[0095] Understandably, the flicker period is used to control the accuracy of the line scan image. Given a fixed longitudinal scanning accuracy of the line scan camera and the moving speed of the workpiece, a smaller flicker period means more information is captured per unit distance the workpiece moves, resulting in a more accurate image. Conversely, a larger flicker period means less information is captured per unit distance, leading to lower accuracy. When the longitudinal scanning accuracy of the line scan camera is fixed, its shooting range is also fixed. By considering the moving speed of the workpiece, we can determine when it will enter the camera's shooting range and control the line scan light source matrix to project cosine light, thus obtaining the corresponding line scan image. The relationship between the flicker period and the longitudinal scanning accuracy and the moving speed of the workpiece can be linear or periodic, depending on the specific circumstances and accuracy requirements; there are no strict limitations on this.

[0096] It is important to understand that the relationship between the scintillation period of the line scan light source matrix, the longitudinal scanning accuracy of the line scan camera, and the moving speed of the workpiece under test is a mathematical relationship.

[0097] In a specific example, a schematic diagram of the line scan camera 10 capturing images on a production line is shown below. Figure 2As shown, the workpiece 20 under test moves along the production line. When the workpiece 20 enters the shooting range of the line scan camera 10, the line scan light source matrix 40 is controlled to project cosine light r1 and r2 with different phases. When the cosine light is projected onto the uneven defect 30 on the workpiece 20, the phase of the vicinity of the workpiece 20 will have a large deviation. The cosine light r1 and r2 are reflected on the uneven defect 30 of the workpiece 20, resulting in r3 and r4, which enter the line scan camera 10. Based on this phase difference Δφ, relevant information about the uneven defect 30 on the workpiece 20 is extracted. It can be understood that by linking the line scan camera 10 and the line scan light source matrix 40, i.e., when the line scan light source matrix 40 projects cosine light, it sends a synchronization signal to the line scan camera 10 for shooting. The line scan camera 10 captures the pattern illuminated by the cosine light on the workpiece 20, obtaining a line scan image, as shown. Figure 3 As shown.

[0098] In a specific example, the arrangement of light sources in the line scan light source matrix is ​​as follows: Figure 4 As shown, 1 represents the light source combination in each row, which is light in the Y direction; 2 represents a single light source in the line scan light source matrix; and 3 represents the light source combination in each column, which is light in the X direction. To achieve phase image capture in high-speed line scan mode, a line scan light source matrix capable of high-speed cosine light projection is proposed. The line scan light source matrix consists of arranged LED beads, with an acrylic plate covering its surface during installation to diffuse the light emitted by the beads and optimize imaging effects. The light source in the X direction needs to present a sine wave in the horizontal direction; the X direction is consistent with the row direction of the image captured by the line scan camera, and the Y direction is consistent with the column direction of the image.

[0099] Controlling the light sources in a linear scan light source matrix requires separate control of the x and y directions. For example, rows 1, 3, 5, and 7 provide light in the y direction, and each row is controlled together. The other LEDs provide light in the x direction, and each column is controlled together. The light source intensity of each row or column conforms to the brightness relationship of a cosine light source.

[0100] In a specific example, after the workpiece is moved a specified distance (e.g., 0.1 mm), the light source in the line scan light matrix illuminates it 8 times, and the line scan camera captures 8 rows of images. This process is repeated after the workpiece moves again. These 8 rows of line scan images correspond to the imaging results of the line scan light matrix illuminating the workpiece 8 times. Figure 5 As shown, the eight images illuminated by the linear scan light source matrix are the four phase images in the X direction in the first row and the four phase images in the Y direction in the second row.

[0101] In order to accurately project cosine light and achieve precise control over the brightness of each row or column of light sources, high-performance embedded devices are used to send PWM signals to each row and column of light sources, thereby enabling precise and high-speed control of the overall brightness.

[0102] Furthermore, the line scan light source matrix includes n rows of line scan light sources, each row of which projects a set of light, and is controlled through the following steps:

[0103] Based on the optical characteristics of the workpiece under test, the brightness of the light source for each row or column is determined, where the light source brightness is specifically expressed as:

[0104]

[0105] I i Let A be the light source brightness in the i-th row / column, B be the base brightness, and α be the brightness variation range. α is the initial phase of the sinusoidal fringes. For phase shift;

[0106] Based on the brightness of the light source, output a light source control signal.

[0107] Taking a light source in the x-direction as an example, the image brightness corresponding to its standard light source brightness is specifically expressed as follows:

[0108] ;

[0109] Where I is the pixel grayscale value of the image, and A x B is the mean attribute of the image. x For the specular reflection property of the image, α x The initial phase of the sinusoidal fringes. This refers to phase shift in the four-step phase shifting method. By controlling the PWM duty cycle of each column or row separately, the brightness of the light source in each column or row is controlled, so that corresponding cosine light stripes can be formed after covering with an acrylic plate.

[0110] S102: Based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase maps.

[0111] Furthermore, based on the arrangement of the line scan light source matrix, the line scan image is divided into multiple phase maps, referring to... Figure 6 Specifically, it includes:

[0112] Based on the arrangement of the line scan light source matrix, the number of light source types projected by the line scan light source matrix is ​​given;

[0113] The row number of each row in the line scan image is moduloed by the number of light source types to obtain the phase map number of each row.

[0114] Based on the phase map number and the row number of each row in the line scan image, the rows in the line scan image are recombined to obtain multiple phase maps.

[0115] It's important to understand that during 2.5D line scanning image capture, the workpiece under test on the production line is constantly moving, and the light source is constantly switching. Each row of the resulting image represents the imaging effect under a single cosine light source. Based on this, the line scan image is split, and the phase image for each row is calculated by taking the remainder of the number of rows divided by the number of light source types.

[0116] In a specific example, the line scan image has 64 rows, each numbered from 1 to 64. There are 8 types of light sources. The modulo operation is performed on each row number, resulting in 1 mod 8 = 1, 2 mod 8 = 2, 3 mod 8 = 3, ..., 64 mod 8 = 0. Rows with equal remainders are then concatenated sequentially, i.e., rows numbered 1, 9, 17, 25, 33, 41, 49, and 57 are concatenated together to obtain one phase image. Similarly, eight phase images can be obtained, such as... Figure 7 As shown.

[0117] After splitting the line scan image, eight phase images are obtained. From top to bottom, they represent four phases in the x-direction and four phases in the y-direction.

[0118] S103: Align and solve the phase map to obtain multiple effect maps.

[0119] The phase map is aligned and solved to obtain multiple effect maps, which include at least one of the following: mean map, diffuse map, specular map, gloss ratio map, and shape map. Specifically, they include:

[0120] Based on the phase direction of each phase map, the phase maps are classified to obtain the x-direction phase map set and the y-direction phase map set;

[0121] Based on the x-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the x-direction are analyzed and calculated.

[0122] Based on the y-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the y-direction are analyzed and calculated.

[0123] Differentiate the wrapping phase in the x-direction and the wrapping phase in the y-direction respectively to obtain the gradient in the x-direction and the gradient in the y-direction;

[0124] Add the gradients in the x and y directions to obtain the shape diagram.

[0125] It's important to understand that after obtaining eight phase maps, the wrap phase, diffuse reflection properties, and specular reflection properties in the x and y directions can be calculated using these phase maps. The wrap phase is specifically represented as:

[0126]

[0127] in, Phase at each pixel, I represents the phase of the light source projected onto the phase diagram. i Let N be the grayscale value of the i-th pixel, and N be the number of pixels.

[0128] The mean attribute is the average of the four phase maps in a single direction. The mean attribute C for each point is specifically expressed as follows:

[0129]

[0130] The specular reflection property D represents the ability of each point to reflect light, specifically expressed as:

[0131]

[0132] After calculating the phase, mean attribute, and specular reflection attribute of each pixel, we can obtain the mean map, diffuse reflection map, specular reflection map, gloss ratio map, and shape map. The mean map is an image composed of the mean attribute values ​​of each pixel; the diffuse reflection map is a spectral map formed by the diffuse reflection intensity of light at different wavelengths; the specular reflection map is an image composed of the specular reflection attribute values ​​of each pixel; the gloss ratio map is an image composed of the ratio of the diffuse reflection attribute value to the specular reflection attribute value of each pixel; and the shape map is an image composed of the derivatives enclosing the phase, with the x-direction phase calculated separately for the horizontal and vertical phases. Phase with the y-direction We need to calculate the gradients in the x-direction and y-direction separately, and then add them together. The gradient G in the x-direction is... x The phase derivative of the Sobel operator with respect to the x-direction is specifically expressed as:

[0133]

[0134] gradient G in the y-direction y The phase derivative of the Sobel operator with respect to the y-direction is:

[0135]

[0136] in, For the Sobel operator in the x-direction, Let y be the Sobel operator in the y direction.

[0137] For the problem of existing phase periodicity changes, it is only necessary to add the phase value at the points where the gradient is large. Recalculate.

[0138] By splitting the line scan image, such as Figure 8As shown, from top to bottom, we obtain the mean map, diffuse reflection map, specular reflection map, gloss ratio map, and shape map. Scratches and defects are reflected in the shape map. The grayscale features in the mean map, diffuse reflection map, specular reflection map, and gloss ratio map all indicate dirt or other contaminants. The dirt information is clearly visible in the specular reflection map and gloss ratio map.

[0139] By splitting the line scan image into a regular phase image, and then calculating the mean, diffuse reflection, specular reflection, gloss ratio, and shape map, dirt information in the background can be filtered out, making scratch detection easier.

[0140] S104: Based on the preset template image, analyze the feature correspondence between the template image and the effect image, remove the artifacts in the effect image, and obtain the standard effect image.

[0141] Furthermore, based on the preset template image, the correspondence between the features of the template image and the rendered image is analyzed, artifacts in the rendered image are removed, and a standard rendered image is obtained for reference. Figure 9 Specifically, it includes:

[0142] Extract the template features from the template image and the effect features from the effect image respectively;

[0143] Construct a feature distance matrix based on the positions of each feature point in the template features and effect features;

[0144] Analyze the feature distance matrix to establish the correspondence between template features and effect features;

[0145] Based on the correspondence between template features and effect features, the effect image is aligned with the template image to obtain the aligned effect image;

[0146] By combining the gradient features in the template image, the artifacts in the alignment effect image are filtered out to obtain the standard effect image.

[0147] It is understandable that when inspecting workpieces with complex surface information, there may be abrupt changes in phase at brightness variations, which can introduce artifacts into the shape image. The artifact removal process is described in [reference needed]. Figure 10 First, the good-quality image is used as the template image. Then, the mean map of the template image is calculated, feature points are extracted, and gradient information is calculated. Next, based on the mean map and shape map of the effect image, feature point information is extracted to obtain effect features. These effect features are then matched with the template features of the template image. Subsequently, the shape map of the effect image is subjected to an affine transformation and mapped onto the template image. The corresponding pixels in the effect image are divided by the gradient of the corresponding position in the template image, thereby filtering out artifacts caused by color changes and obtaining the standard effect image. The good-quality image is an image that meets specific quality standards and requirements.

[0148] Furthermore, the feature distance matrix is ​​analyzed to establish the correspondence between template features and effect features, specifically including:

[0149] For each feature point in the effect features, the nearest and second nearest distance values ​​are selected from the feature distance matrix;

[0150] Analyze the distance ratio between the nearest and second-nearest distance values;

[0151] If the distance ratio is within the preset distance range, a correspondence is established with the feature point in the template feature corresponding to the nearest distance value;

[0152] If the distance ratio is not within the preset distance range, discard the nearest distance value and re-determine it until the correspondence between all feature points in the effect feature and the template feature is completed.

[0153] In one specific implementation, the Euclidean distance between each feature point in the effect feature and each feature point in the template feature is calculated, and each element in the feature distance matrix is ​​the Euclidean distance between the i-th feature point in the effect feature and the j-th feature point in the template feature. Each feature point in the effect feature is analyzed sequentially to obtain the nearest and second-nearest distance values ​​corresponding to each feature point. The ratio of the nearest and second-nearest distance values ​​is calculated, i.e., the distance ratio. A smaller distance ratio indicates a larger difference between the nearest and second-nearest distance values, higher distinguishability between the feature points corresponding to the nearest and second-nearest distance values, and thus a more reliable feature point, allowing for the establishment of a correspondence. Conversely, a larger distance ratio indicates a smaller difference between the nearest and second-nearest distance values, lower distinguishability, and thus a less reliable feature point, preventing the establishment of a correspondence. In this case, the feature point corresponding to the nearest distance value is discarded, and a new nearest distance value is determined until the distance ratio falls within a preset range, allowing for the establishment of a correspondence.

[0154] Based on the correspondence between template features and effect features, an affine transformation is performed on the effect image to align it with the template image, resulting in an aligned effect image. After obtaining the aligned effect image, the gradient features of each feature point in the template image are calculated. Dividing each feature point in the effect image by the gradient features of the corresponding position in the template image filters out artifacts in the aligned effect image, yielding the standard effect image.

[0155] The gradient feature of an image is a two-dimensional vector representing the rate of change of image intensity at each pixel location. For a grayscale image I(x,y), the gradient feature... At position (x, y), it can be represented as:

[0156]

[0157] in, Let be the partial derivative of I(x,y) in the horizontal direction. Let I(x,y) be the partial derivative of I(x,y) in the vertical direction.

[0158] Gradient features can be calculated using operators such as Sobel, Prewitt, and Roberts. In this example, the Sobel operator is used. In a specific example, for any pixel value of a feature point in the template image, convolution operations are performed with the Sobel operator's convolution kernels in the x and y directions, respectively, to obtain the x-gradient and y-gradient components. Taking the square root of the sum of the squares of the x-gradient and y-gradient components yields the gradient feature corresponding to that feature point. Similarly, the gradient features of all feature points in the template image can be calculated. By iterating through each feature point in the alignment result image, and for each feature point, dividing its pixel value by the gradient feature of the corresponding feature point in the template image, artifacts in the alignment result image are removed, resulting in the standard result image.

[0159] To address the issue of 2.5D shape artifacts caused by drastic color changes, this invention uses feature point matching between the template image and the test image to map pixels, and then filters the shape image according to the gradient of the template mean image, thereby eliminating artifacts and improving the accuracy of defect detection.

[0160] S105: Compare and classify the defect features in the standard rendering to obtain the defect results and complete the defect detection.

[0161] Furthermore, the defect features in the standard rendering are compared and classified to obtain defect results, thus completing defect detection, and referring to... Figure 11 Specifically, it includes:

[0162] The standard rendering is binarized to enhance defect features, resulting in the first rendering.

[0163] A closing operation is performed on the first effect image to fuse the discrete defect feature points and give the second effect image;

[0164] Based on the starting point of the defect feature region in the second rendering, determine the minimum bounding rectangle of the defect feature region;

[0165] Analyze the aspect ratio of the minimum bounding rectangle and provide the corresponding defect results.

[0166] Furthermore, a closing operation is performed on the first effect image to fuse the discrete defect feature points, resulting in a second effect image, specifically including:

[0167] Based on a pre-defined unit of operation, the unit of operation is traversed in the foreground of the first effect image. If the unit of operation includes the foreground, all pixel values ​​in the unit of operation are switched to the foreground, and the foreground in the first effect image is expanded to obtain an expanded effect image.

[0168] The operation unit is traversed in the foreground of the expansion effect image. If the operation unit includes the background, the center of the operation unit is switched to the background, and the foreground of the expansion effect image is eroded to obtain the second effect image.

[0169] Understandably, after calculating the standard effect image, a binarization operation is performed on the standard effect image. However, due to issues such as shooting noise and angle, a defect may be discontinuous. Therefore, performing a closing operation on the standard effect image can connect the defect parts together and give the minimum bounding rectangle of the defect, displaying the defect detected in the line scan image and obtaining the defect result.

[0170] Understandably, after calculating the standard effect image, a binarization operation is performed on the standard effect image. However, due to issues such as shooting noise and angle, a defect may be discontinuous. Therefore, performing a closing operation on the standard effect image can connect the defect parts together and give the minimum bounding rectangle of the defect, displaying the defect detected in the line scan image and obtaining the defect result.

[0171] The aforementioned preset operational unit is a pixel block composed of multiple pixels. It can be a circle with radius r, a cross shape with length L, or a rectangle; there is no limitation in this regard. Each operational unit has a center; the center of the circle is the center of the circle, and the center point of the cross shape is the intersection of the horizontal and vertical axes. In one specific embodiment, the center of the operational unit is traversed through the foreground of the first effect image. If the operational unit includes the foreground, all pixel values ​​in the operational unit are switched to the foreground, and the foreground in the first effect image is expanded to obtain an expanded effect image. For example, if the operational unit consists of 5 pixel values, during the traversal, the center of the operational unit passes through each pixel in the foreground in turn, switching the background pixel values ​​covered by other operational units to the foreground. After completing the traversal, an expanded effect image is obtained. Then, the center of the operational unit is traversed through the foreground of the expanded effect image. If the operational unit includes the background, the center of the operational unit is switched to the background, and the foreground of the expanded effect image is eroded to obtain a second effect image. For example, during the traversal, if one of the five pixel values ​​of the operation unit is the background, then the center of the operation unit is switched to the background. After completing the traversal, the second effect image is obtained.

[0172] After obtaining the second rendering, a starting point is determined from the foreground. This can be the point with the smallest or largest x and y coordinates, or another edge point of the foreground. Rotating from this starting point with the horizontal direction as the starting point, the points tangent to the foreground are recorded during the rotation. For each tangent point, a bounding rectangle is constructed, and the area of ​​each bounding rectangle is analyzed. The bounding rectangle with the smallest area is determined as the minimum bounding rectangle. The aspect ratio of the minimum bounding rectangle is then analyzed, and defect features are classified based on the aspect ratio. Defects with an aspect ratio close to 1 are classified as convex hull or pit features, while those with an aspect ratio reaching a preset scratch threshold are classified as scratch information, thus obtaining the corresponding defect results.

[0173] The proposed 2.5D defect detection method based on line scanning imaging provides a complete solution from setting the line scanning light source matrix to analyzing and processing the line scanning images. It is applicable to continuous production scenarios. Compared with traditional 2.5D detection, this invention has a wider range of applications, better stability, and lower light source cost.

[0174] Reference Figure 12 This invention provides a 2.5D defect detection device based on line scan imaging, comprising:

[0175] Image acquisition module 201 is used to acquire line scan images, wherein the line scan image is the workpiece to be measured captured by the line scan camera and illuminated by the line scan light source matrix, and the light source in the line scan light source matrix is ​​used to project cosine light.

[0176] Image segmentation module 202 is used to segment a line scan image into multiple phase images based on the arrangement of the line scan light source matrix;

[0177] Phase resolution module 203 is used to align and resolve the phase map to obtain multiple effect maps;

[0178] The artifact removal module 204 is used to analyze the feature correspondence between the template image and the effect image based on the preset template image, remove artifacts in the effect image, and obtain a standard effect image.

[0179] The defect identification module 205 is used to compare and classify the defect features in the standard effect image to obtain the defect results and complete the defect detection.

[0180] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0181] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and variations of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A 2.5D defect detection method based on line scan imaging, characterized in that, include: Acquire line scan images, where the line scan image is the workpiece to be measured captured by the line scan camera and illuminated by the line scan light source matrix. The light source in the line scan light source matrix is ​​used to project cosine light. Based on the arrangement of the line scan light source matrix, the line scan image is split into multiple phase maps; The phase map is aligned and solved to obtain multiple effect maps; Based on a preset template image, the feature correspondence between the template image and the effect image is analyzed, and artifacts in the effect image are removed to obtain a standard effect image. The artifact removal includes aligning the effect image with the template image based on the correspondence between template features and effect features to obtain an aligned effect image; and filtering artifacts in the aligned effect image by combining gradient features in the template image. The defect features in the standard rendering are compared and classified to obtain the defect results, thus completing the defect detection.

2. The 2.5D defect detection method based on line scan imaging as described in claim 1, characterized in that, The line-scan light source matrix consists of n rows of line-scan light sources. Each row or column of line-scan light sources projects a set of light. The light sources in the line-scan light source matrix are used to project cosine light, and the projection of cosine light is controlled through the following steps: Based on the optical characteristics of the workpiece under test, the brightness of the light source for each row or column is determined, where the light source brightness is specifically expressed as: ； I i Let A be the light source brightness in the i-th row / column, B be the base brightness, α be the initial phase of the sinusoidal stripes, and φ be the phase offset. Based on the brightness of the light source, output a light source control signal.

3. The 2.5D defect detection method based on line scan imaging as described in claim 1, characterized in that, Based on the arrangement of the line scan light source matrix, the line scan image is divided into multiple phase maps, specifically including: Based on the arrangement of the line scan light source matrix, the number of light source types projected by the line scan light source matrix is ​​given; The row number of each row in the line scan image is moduloed by the number of light source types to obtain the phase map number of each row. Based on the phase map number and the row number of each row in the line scan image, the rows in the line scan image are recombined to obtain multiple phase maps.

4. The 2.5D defect detection method based on line scan imaging as described in claim 1 or 3, characterized in that, The renderings include at least one of the following: mean plot, diffuse plot, specular plot, gloss ratio plot, and shape plot; The phase map is aligned and solved to obtain multiple result images, including: Based on the phase direction of each phase map, the phase maps are classified to obtain the x-direction phase map set and the y-direction phase map set; Based on the x-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the x-direction are analyzed and calculated. Based on the y-direction phase atlas, the wrapping phase, diffuse reflection attribute value, and specular reflection attribute value in the y-direction are analyzed and calculated. Differentiate the wrapping phase in the x-direction and the wrapping phase in the y-direction respectively to obtain the gradient in the x-direction and the gradient in the y-direction; Add the gradients in the x and y directions to obtain the shape diagram.

5. The 2.5D defect detection method based on line scan imaging as described in claim 1, characterized in that, Based on the preset template image, analyze the feature correspondence between the template image and the rendered image, remove artifacts from the rendered image, and obtain the standard rendered image, specifically including: Extract the template features from the template image and the effect features from the effect image respectively; Construct a feature distance matrix based on the positions of each feature point in the template features and effect features; Analyze the feature distance matrix to establish the correspondence between template features and effect features; Based on the correspondence between template features and effect features, the effect image is aligned with the template image to obtain the aligned effect image; By combining the gradient features in the template image, the artifacts in the alignment effect image are filtered out to obtain the standard effect image.

6. The 2.5D defect detection method based on line scan imaging as described in claim 5, characterized in that, The feature distance matrix is ​​analyzed to establish the correspondence between template features and effect features, specifically including: For each feature point in the effect features, the nearest and second nearest distance values ​​are selected from the feature distance matrix; Analyze the distance ratio between the nearest and second-nearest distance values; If the distance ratio is within the preset distance range, a correspondence is established with the feature point in the template feature corresponding to the nearest distance value; If the distance ratio is not within the preset distance range, discard the nearest distance value and re-determine it until the correspondence between all feature points in the effect feature and the template feature is completed.

7. The 2.5D defect detection method based on line scan imaging as described in claim 1, characterized in that, The defect features in the standard rendering are compared and classified to obtain defect results, thus completing defect detection. This process specifically includes: The standard rendering is binarized to enhance defect features, resulting in the first rendering. A closing operation is performed on the first effect image to fuse the discrete defect feature points and give the second effect image; Based on the starting point of the defect feature region in the second rendering, determine the minimum bounding rectangle of the defect feature region; Analyze the aspect ratio of the minimum bounding rectangle and provide the corresponding defect results.

8. The 2.5D defect detection method based on line scan imaging as described in claim 7, characterized in that, A closing operation is performed on the first effect image to fuse the discrete defect feature points, resulting in a second effect image, which specifically includes: Based on a pre-defined unit of operation, the unit of operation is traversed in the foreground of the first effect image. If the unit of operation includes the foreground, all pixel values ​​in the unit of operation are switched to the foreground, and the foreground in the first effect image is expanded to obtain an expanded effect image. The operation unit is traversed in the foreground of the expansion effect image. If the operation unit includes the background, the center of the operation unit is switched to the background, and the foreground of the expansion effect image is eroded to obtain the second effect image.

9. The 2.5D defect detection method based on line scan imaging as described in claim 1, characterized in that, Controlling the light sources in the line scan light source matrix includes controlling the light sources in the X direction and the light sources in the Y direction. Specifically, controlling the light sources in each row of the line scan light source matrix is ​​controlling the light sources in the Y direction, and controlling the light sources in each column of the line scan light source matrix is ​​controlling the light sources in the X direction. The scintillation period of the line scan light source matrix is ​​determined based on the longitudinal scanning accuracy of the line scan camera and the moving speed of the workpiece under test. Based on the flicker period, the light sources in each row or column of the line scan light source matrix are controlled separately.

10. A 2.5D defect detection device based on line scan imaging, characterized in that, The 2.5D defect detection method based on line scanning imaging as described in any one of claims 1-9 includes: The image acquisition module is used to acquire line scan images, wherein the line scan image is the workpiece under test captured by the line scan camera and illuminated by the line scan light source matrix, and the light source in the line scan light source matrix is ​​used to project cosine light; The image splitting module is used to split the line scan image into multiple phase maps based on the arrangement of the line scan light source matrix; The phase resolution module is used to align and resolve the phase map to obtain multiple effect maps; The artifact removal module is used to analyze the feature correspondence between the template image and the effect image based on a preset template image, remove artifacts in the effect image, and obtain a standard effect image. The artifact removal includes aligning the effect image with the template image based on the correspondence between template features and effect features to obtain an aligned effect image; and filtering artifacts in the aligned effect image by combining gradient features in the template image. The defect identification module is used to compare and classify the defect features in the standard rendering to obtain the defect results and complete the defect detection.

Images