Defect detection method, system and device

Abstract

The embodiment of the invention provides a defect detection method, system and device, and relates to the technical field of defect detection.The method comprises the steps that a light source is controlled to project a projection pattern of at least one target type according to the surface attribute of a to-be-detected object, and a camera is controlled to collect a corresponding projection image; processing each projection image by using an image processing algorithm corresponding to each target type of projection pattern to obtain a first defect detection result; and fusing the first defect detection results to obtain a target defect detection result. According to the method provided by the embodiment of the invention, no matter the surface attributes of the to-be-measured object are one or more, the corresponding projection pattern is selected according to the surface attributes of the to-be-measured object, the camera is controlled to acquire the corresponding projection image, and then the image processing algorithm corresponding to the projection pattern is used for processing; therefore, the defect detection result of the to-be-detected object can be obtained without making a detection scheme for the surface attribute of the to-be-detected object, and the defect detection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a defect detection method, system and device. Background Art

[0002] In industries such as automobile manufacturing, electronic manufacturing, and medical devices, product appearance defects directly affect product quality. Therefore, it is necessary to conduct defect detection on the product surface during the production process to promptly discover and repair potential problems. Surface defects can be detected by light source lighting inspection, machine vision inspection, instrument inspection, magnetic particle inspection, and penetrant flaw detection. In industrial production, in order to realize automated defect detection, machine vision inspection is usually used to detect surface defects of products.

[0003] When using machine vision to detect surface defects, a light source with a certain pattern is projected onto the object to be tested, a camera is used to capture the surface image of the object to be tested, and then an image processing algorithm is used to analyze and process the image to identify and locate the defects. Currently, a light source is usually used to generate a specific stripe pattern and project it onto the object to be tested. The defective area and the normal area on the object to be tested will have obvious differences in the stripe pattern. These differences are captured by image acquisition devices such as cameras and converted into digital image signals.

[0004] In actual application scenarios, the surface properties of the objects to be tested are diverse, and the surface properties of different objects to be tested are also different. It is necessary to formulate corresponding detection plans according to the surface properties of each object to be tested, resulting in low defect detection efficiency. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a defect detection method, system and device to realize defect detection of objects with different surface properties. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, a defect detection method is provided, the method comprising:

[0007] Controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type; wherein the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0008] Processing each of the projection images using an image processing algorithm corresponding to the projection pattern of each of the target types to obtain a first defect detection result;

[0009] The first defect detection results are fused to obtain a target defect detection result.

[0010] In a possible implementation manner, the using an image processing algorithm corresponding to the projection pattern of each target type to process each projection image to obtain a first defect detection result includes:

[0011] According to the type of the projection pattern, each projection image is divided into a plurality of image groups of target types, wherein the type of the projection pattern in each projection image in the same image group is the same, and the type of the projection pattern in each projection image in different image groups is different;

[0012] For each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

[0013] In a possible implementation, controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured includes:

[0014] Obtaining a surface attribute selection instruction for the object to be measured, and controlling a light source to project a projection pattern of a target type corresponding to the surface attribute indicated by the surface attribute selection instruction; wherein the correspondence between the surface attribute indicated by the surface attribute selection instruction and the type of the projection pattern is pre-configured;

[0015] or,

[0016] In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, controlling the light source to project a projection pattern of at least one target type indicated by the pattern selection instruction;

[0017] or,

[0018] Acquire the surface properties of the object to be measured obtained by identifying the first image captured by the camera, and control the light source to project a projection pattern of a target type corresponding to the surface properties, wherein the correspondence between the surface properties and the type of projection pattern is pre-configured.

[0019] In a possible implementation manner, the correspondence between the surface attribute and the type of the projection pattern is pre-configured in the following manner:

[0020] In response to the relationship establishment instruction, determining the surface attribute indicated by the relationship establishment instruction and the type of the projection pattern indicated by the relationship establishment instruction;

[0021] A correspondence between the surface attribute indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

[0022] In a possible implementation manner, the projection pattern of the same target type includes a plurality of sub-patterns, and different projection areas are arranged on the light source, and the positions of the projection areas relative to the object to be measured are different;

[0023] The image processing algorithm includes a photometric stereo method, and each sub-pattern included in a projection pattern corresponding to the photometric stereo method is projected onto the object to be measured from different projection areas.

[0024] In a possible implementation manner, the controlling the camera to collect a projection image corresponding to the projection pattern of each target type includes:

[0025] Controlling the camera to collect a preset number of image frames respectively to obtain original images, wherein the projection pattern projected on the object to be measured in each image frame is different;

[0026] The original image is split according to the types of the projection patterns to obtain projection images corresponding to the projection patterns of each target type.

[0027] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the method further includes:

[0028] Each sub-pattern is numbered based on the type of projection pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0029] The controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type, include:

[0030] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0031] If i is less than M, then P i+1，j As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern;

[0032] If i is equal to M, then P i，j+1 As the new current sub-pattern, return to execute the i，jAs the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

[0033] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the method further includes:

[0034] Each sub-pattern is numbered based on the type of projection pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0035] The controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type, include:

[0036] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern;

[0037] If j is less than N, then P i，j+1 As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern;

[0038] If j is equal to N, then P i+1，j As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

[0039] According to a second aspect of an embodiment of the present application, a defect detection system is provided, the defect detection system comprising a light source, a camera, a first controller and a second controller;

[0040] The light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0041] The camera is used to collect the projected image;

[0042] The first controller is used to control the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and to control the camera to collect projection images corresponding to the projection patterns of each target type;

[0043] The second controller is used to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result; and to fuse each of the first defect detection results to obtain a target defect detection result.

[0044] In a possible implementation manner, the second controller is specifically configured to:

[0045] According to the type of the projection pattern, each projection image is divided into a plurality of image groups of target types, wherein the type of the projection pattern in each projection image in the same image group is the same, and the type of the projection pattern in each projection image in different image groups is different;

[0046] For each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

[0047] In a possible implementation manner, the first controller is specifically configured to:

[0048] Obtaining a surface attribute selection instruction for the object to be measured, and controlling a light source to project a projection pattern of a target type corresponding to the surface attribute indicated by the surface attribute selection instruction; wherein the correspondence between the surface attribute indicated by the surface attribute selection instruction and the type of the projection pattern is pre-configured;

[0049] or,

[0050] In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, controlling the light source to project a projection pattern of at least one target type indicated by the pattern selection instruction;

[0051] or,

[0052] Acquire the surface properties of the object to be measured obtained by identifying the first image captured by the camera, and control the light source to project a projection pattern of a target type corresponding to the surface properties, wherein the correspondence between the surface properties and the type of projection pattern is pre-configured.

[0053] In a possible implementation manner, the correspondence between the surface attribute and the type of the projection pattern is pre-configured in the following manner:

[0054] In response to the relationship establishment instruction, determining the surface attribute indicated by the relationship establishment instruction and the type of the projection pattern indicated by the relationship establishment instruction;

[0055] A correspondence between the surface attribute indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

[0056] In a possible implementation manner, the projection pattern of the same target type includes a plurality of sub-patterns, and different projection areas are arranged on the light source, and the positions of the projection areas relative to the object to be measured are different;

[0057] The image processing algorithm includes a photometric stereo method, and each sub-pattern included in a projection pattern corresponding to the photometric stereo method is projected onto the object to be measured from different projection areas.

[0058] In a possible implementation manner, the first controller is specifically configured to:

[0059] Controlling the camera to collect a preset number of image frames respectively to obtain original images, wherein the projection pattern projected on the object to be measured in each image frame is different;

[0060] The original image is split according to the types of the projection patterns to obtain projection images corresponding to the projection patterns of each target type.

[0061] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the system further includes:

[0062] A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0063] The first controller is specifically used for:

[0064] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern;

[0065] If i is less than M, then P i+1，j As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern;

[0066] If i is equal to M, then P i，j+1 As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

[0067] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the system further includes:

[0068] A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0069] The first controller is specifically used for:

[0070] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern;

[0071] If j is less than N, then P i，j+1 As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern;

[0072] If j is equal to N, then P i+1，j As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

[0073] According to a third aspect of the present application, a defect detection device is provided, the device comprising:

[0074] A control module, used to control the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and to control the camera to collect projection images corresponding to the projection patterns of each target type; wherein the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0075] A processing module, configured to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result;

[0076] A fusion module is used to fuse the first defect detection results to obtain a target defect detection result.

[0077] A fourth aspect of the embodiments of the present application further provides an electronic device, including:

[0078] Memory, used to store computer programs;

[0079] The processor is used to implement any of the above-mentioned defect detection methods when executing the program stored in the memory.

[0080] A fifth aspect of an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned defect detection methods is implemented.

[0081] Beneficial effects of the embodiments of the present application:

[0082] The defect detection method, system and device provided in the embodiments of the present application control the light source to project at least one target type of projection pattern according to the surface properties of the object to be tested, and control the camera to collect projection images corresponding to the projection patterns of each target type; and use the image processing algorithm corresponding to the projection pattern of each target type to process each projection image to obtain a first defect detection result; finally, fuse each first defect detection result to obtain a target defect detection result. Since different types of projection patterns and their corresponding image processing algorithms can detect different types of defects, when performing defect detection on the object to be tested, no matter whether the surface properties of the object to be tested are one or more, it is only necessary to select the corresponding projection pattern according to the surface properties of the object to be tested, and control the camera to collect the corresponding projection image, and then use the image processing algorithm corresponding to the projection pattern to process the image, so as to obtain the defect detection result of the object to be tested, without formulating a corresponding detection plan for the surface properties of the object to be tested, and the defect detection of the product to be tested can be quickly performed, thereby improving the efficiency of defect detection.

[0083] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0085] Figure 1 A first schematic diagram of a defect detection method provided in an embodiment of the present application;

[0086] Figure 2a A schematic diagram of projecting a pattern using an angled projection method provided in an embodiment of the present application;

[0087] Figure 2b A schematic diagram of projecting a pattern using coaxial reflection provided in an embodiment of the present application;

[0088] Figure 2c A schematic diagram of projecting a pattern in a transmission manner provided in an embodiment of the present application;

[0089] Figure 3 A first example diagram of a sinusoidal fringe projection pattern provided in an embodiment of the present application;

[0090] Figure 4 A second example diagram of a sinusoidal fringe projection pattern provided in an embodiment of the present application;

[0091] Figure 5 An example diagram of binary stripes provided in an embodiment of the present application;

[0092] Figure 6 for Figure 2a A first principle schematic diagram of the defect detection method of the illustrated embodiment;

[0093] Figure 7 A schematic diagram of the relative position change between the object to be measured and the projection pattern when the object to be measured moves provided in an embodiment of the present application;

[0094] Figure 8 for Figure 7 An example diagram of an image of the surface of the object to be measured in the illustrated embodiment;

[0095] Fig. 9 for Figure 2a A second principle schematic diagram of the defect detection method of the illustrated embodiment;

[0096] Fig.10 for Fig. 9 Equivalent schematic diagram of ;

[0097] Fig.11 A second schematic diagram of the defect detection method provided in an embodiment of the present application;

[0098] Fig.12a A schematic diagram of an original image provided in an embodiment of the present application;

[0099] Figure 12b A first schematic diagram of a split original image provided in an embodiment of the present application;

[0100] Fig.12c A second schematic diagram of the split original image provided in an embodiment of the present application;

[0101] Fig.13a A first flow chart of a controller provided in an embodiment of the present application controlling an image acquisition device to photograph an object to be measured;

[0102] Fig.13bA second flow chart of a controller provided in an embodiment of the present application controlling an image acquisition device to photograph an object to be measured;

[0103] Fig.14 Shown is a third schematic diagram of the defect detection method provided in an embodiment of the present application;

[0104] Fig.15 A schematic diagram of the structure of a defect detection system provided in an embodiment of the present application;

[0105] Fig.16 A schematic diagram of the structure of a defect detection device provided in an embodiment of the present application;

[0106] Fig.17 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0107] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0108] First, the professional terms appearing in the embodiments of the present application are explained:

[0109] Programmable stripe light source: a light source that can output a variety of stripe spots through program control, such as sine or cosine stripes of different widths and phase changes, binary stripes, and stripes of different colors and brightness;

[0110] Phase polarization method: a method for defect measurement based on the sine / cosine stripe light spot reflected from the object surface. The principle is that if there are defects on the surface of a mirror object, the reflected stripes will have different phase gradient changes. According to the phase gradient collected by the camera, the defects that are difficult for the human eye to distinguish can be clearly presented. At the same time, because it is only related to the reflection characteristics of the object surface, it is less affected by the color of the object surface and has a strong interference-free characteristic.

[0111] Photometric stereo method: a method for estimating surface defects of an object by obtaining images of its surface by illuminating it from different directions.

[0112] In the related art, different defect detection algorithms can be used when machine vision detection is used for defect detection, such as photometric stereo, phase deflection method, etc. When the phase deflection method is used, the display device and the camera are usually placed symmetrically, and the center lines of the two intersect at the surface of the object to be measured; when the photometric stereo method is used, it is necessary to collect lighting effect images at different angles for the same position of the object to be measured, so it is necessary to illuminate the object to be measured from light sources at different angles. When the phase deflection method is used to detect defects on the surface of an object, if the mirror reflection of the object surface is small, the detection result of the phase deflection method is inaccurate, so the phase deflection method is usually used for defect detection on mirror reflection surfaces; when the photometric stereo method is used to detect defects on the surface of an object, if the mirror reflection of the object surface is high, the measurement result error of the photometric stereo method is large, so the photometric stereo method is usually used for defect detection on diffuse reflection surfaces.

[0113] However, in actual application scenarios, the surface properties of the objects to be tested are diverse, not just smooth surfaces or diffuse reflection surfaces. For objects with complex surface properties, the above method cannot detect all surface defects of the objects to be tested. It is necessary to formulate a corresponding defect detection plan based on the surface properties of the objects to be tested. Since the surface properties of different objects to be tested may be different, it is necessary to formulate a corresponding defect detection plan for each surface property of the objects to be tested, resulting in low defect detection efficiency.

[0114] In order to achieve rapid defect detection on different objects to be tested and thereby improve the efficiency of defect detection, the embodiments of the present application provide a defect detection method, system and device. The defect detection method provided by the present application is described in detail below through specific embodiments.

[0115] In a first aspect of the embodiments of the present application, a defect detection method is first provided. Figure 1 , Figure 1 The first flow chart of the defect detection method provided in the embodiment of the present application specifically includes the following steps:

[0116] Step S10, controlling the light source to project a projection pattern of at least one target type according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type;

[0117] Wherein, the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0118] Step S20, processing each projection image using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result;

[0119] Step S30, fusing the first defect detection results to obtain a target defect detection result.

[0120] Through the method of the embodiment of the present application, since different types of projection patterns and their corresponding image processing algorithms can detect different types of defects, when performing defect detection on the object to be tested, no matter whether the surface properties of the object to be tested are one or more, it is only necessary to select the corresponding projection pattern according to the surface properties of the object to be tested, and control the camera to capture the corresponding projection image, and then use the image processing algorithm corresponding to the projection pattern to process the image, so as to obtain the defect detection result of the object to be tested, without formulating a corresponding detection plan according to the surface properties of the object to be tested. The defect detection of the product to be tested can be quickly performed, thereby improving the defect detection efficiency.

[0121] The above steps S10 to S30 are actually composed of four processes: pattern projection process, image acquisition process, image processing process and defect detection process, wherein step S10 is actually the pattern projection process and image acquisition process, step S20 is the image processing process, and step S30 is the defect detection process. The above steps S10 to S30 are described below in conjunction with the accompanying drawings.

[0122] First, the pattern projection process and image acquisition process are explained:

[0123] The light source may be a programmable stripe light source or a display device that can display a preset pattern. In a possible implementation, the light source wavelength band includes visible light and infrared light with a wavelength of 850 nanometers and infrared light with a wavelength of 940 nanometers.

[0124] The material type of the surface of the object to be tested determines the reflectivity. A material with a smooth surface will reflect more incident light, a material with a darker surface will absorb more incident light, and a material with a rough surface will scatter the reflected light in all directions. In this application, the material type of the surface of the object to be tested is referred to as the surface property of the object to be tested.

[0125] Since the light source characteristics of the light source, the shape and size of the pattern projected by the light source, and the surface properties of the object to be measured will all affect the reflectivity of the object surface, for the sake of ease of description, in this application, the patterns with different light source characteristics, pattern shapes, and sizes projected by the light source are referred to as different types of projection patterns. That is, different types of projection patterns in this application refer to different light source characteristics, generation methods, and waveform characteristics. For example, sine fringes, cosine fringes, binary fringes, and bright block patterns are different types of projection patterns.

[0126] It can be understood that the light source in the embodiment of the present application is used to project at least two different types of projection patterns corresponding to the surface properties of different objects to be tested, which means that the light source can project at least two different types of projection patterns corresponding to the surface properties of different objects to be tested, and does not mean that the light source must project at least two different types of projection patterns during defect detection. The type of projection pattern projected by the light source depends on the surface properties of the object to be tested. For example, if the surface of the object A to be tested is a smooth surface, then there is only one surface property of the surface A of the object to be tested, and then when the object A to be tested is subjected to defect detection, the light source is controlled to project a projection pattern corresponding to the smooth surface; for another example, if the surface of the object B to be tested is a smooth surface and a rough surface, then there are two surface properties of the object B to be tested, and then when the object B to be tested is subjected to defect detection, the light source is controlled to project two types of projection patterns, namely, a projection pattern corresponding to the smooth surface and a projection pattern corresponding to the rough surface.

[0127] In the above step S10, when controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, the following three situations may be specifically included:

[0128] In the first case, a correspondence between surface attributes and the type of projection pattern is pre-established, and the user selects the surface attribute, and controls the light source to project the projection pattern corresponding to the surface attribute selected by the user. Based on this, in a possible implementation, the above step S10 may specifically include:

[0129] A surface property selection instruction for the object to be measured is obtained, and a light source is controlled to project a projection pattern of a target type corresponding to the surface property indicated by the surface property selection instruction.

[0130] For example, assuming that the surface attribute of the object 2 to be measured is surface attribute 1, the surface attribute indicated by the surface attribute selection instruction input by the user is surface attribute 1. Since the correspondence between surface attribute 1 and projection pattern type 1 is pre-established, the light source can be controlled to project the projection pattern of projection pattern type 1. For another example, assuming that the surface attributes of the object 2 to be measured are surface attributes 2 and surface attributes 3, the surface attributes indicated by the surface attribute selection instruction input by the user are: surface attribute 2 and surface attribute 3. Since the correspondence between surface attribute 2 and projection pattern type 2 and the correspondence between surface attribute 3 and projection pattern 3 are pre-established, the light source can be controlled to project the projection pattern of projection pattern type 2 and the projection pattern of projection pattern type 3, respectively.

[0131] In the second case, a correspondence between surface attributes and the type of projection pattern is established in advance, and the first image of the object to be measured captured by the camera is recognized to determine the surface attributes of the object to be measured, and then the light source is controlled to project a projection pattern of a target type corresponding to the surface attributes. Based on this, in a possible implementation, the above step S10 may specifically include:

[0132] The surface properties of the object to be measured are obtained by identifying the first image captured by the camera, and the light source is controlled to project a projection pattern of a target type corresponding to the surface properties.

[0133] It can be understood that the surface properties of the object to be tested are obtained by identifying the first image captured by the camera. Before defect detection is performed on the object to be tested, the camera can be controlled to capture the first image of the object to be tested and the first image of the object to be tested can be identified. Alternatively, the camera can be controlled to capture the first image of each object to be tested and the first image of each object to be tested can be identified in advance. The embodiments of the present application are not limited to this.

[0134] Taking the example of controlling the camera to capture the first image of the object to be tested 2 and identifying the first image of the object to be tested 2 to obtain the surface properties of the object to be tested 2 before defect detection is performed on the object to be tested 2, assuming that the surface property of the object to be tested 2 is surface property 1, since the correspondence between surface property 1 and projection pattern type 1 is pre-established, the light source can be controlled to project the projection pattern of projection pattern type 1; for another example, assuming that the surface properties of the object to be tested 2 are surface property 2 and surface property 3, since the correspondence between surface property 2 and projection pattern type 2 and the correspondence between surface property 3 and projection pattern type 3 are pre-established, the light source can be controlled to project the projection pattern of projection pattern type 2 and the projection pattern of projection pattern type 3, respectively.

[0135] The correspondence between the surface attributes and the projection pattern in the above case 1 and case 2 is pre-configured. In a possible implementation, the correspondence between the surface attributes and the projection pattern is established in the following manner:

[0136] In response to the relationship establishment instruction, the surface attributes indicated by the relationship establishment instruction and the type of the projection pattern indicated by the relationship establishment instruction are determined; and a correspondence between the surface attributes indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

[0137] The relationship establishment instruction includes the surface attribute and the type of projection pattern that the user wants to establish a corresponding relationship. Exemplarily, assuming that no corresponding relationship between any surface attribute and the type of projection pattern has been established, and the user wants to establish a corresponding relationship between surface attribute 1 and projection pattern type 1, the user inputs a relationship establishment instruction, and in response to the relationship establishment instruction input by the user, it is determined that the surface attribute indicated by the relationship establishment instruction is surface attribute 1, and the projection pattern indicated by the relationship establishment instruction is a projection pattern of projection pattern type 1, and the corresponding relationship between surface attribute 1 and projection pattern type 1 is established.

[0138] By using the embodiments of the present application, a user can configure a suitable projection pattern according to the surface properties of the object to be tested, thereby ensuring that defects on the surface of the object to be tested can be accurately and effectively detected, thereby improving the accuracy of defect detection.

[0139] Case 3: The user directly selects a projection pattern, and the light source is controlled to project the projection pattern selected by the user. Based on this, in a possible implementation, the above step S10 may specifically include:

[0140] In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, the light source is controlled to project a projection pattern of at least one target type indicated by the pattern selection instruction.

[0141] Among them, the pattern selection instruction is input by the user according to the surface properties of the object to be measured. For example, assuming that the surface property of the object to be measured is surface property 1, and the projection pattern corresponding to surface property 1 is the projection pattern of projection pattern type 1, then the projection pattern indicated by the pattern selection instruction is the projection pattern of projection pattern type 1; for another example, assuming that the surface properties of the object to be measured are surface properties 1 and surface properties 2, the projection pattern corresponding to surface property 1 is the projection pattern of projection pattern type 1, and the projection pattern corresponding to surface property 2 is the projection pattern of projection pattern type 2, then the projection pattern indicated by the pattern selection instruction is the projection pattern of projection pattern type 1 and the projection pattern of projection pattern type 2.

[0142] In the pattern projection process and the image acquisition process, the pattern projection mode is different according to the relative position of the light source 1 and the image acquisition device 3. The pattern projection mode includes angled reflection, coaxial reflection, transmission, etc. Figure 2aThe diagram is a schematic diagram with angled reflection, where the light source 1 is placed on one side of the object to be tested 2, and the image acquisition device 3 is placed on the other side of the object to be tested 2. An angle α exists between the optical axis of the light source 1 and the lens optical axis of the image acquisition device 3. When performing defect detection, the device 1 controls the light source 1 to project a first projection pattern onto the object to be tested 2, and controls the image acquisition device 3 to capture the image of the object to be tested 2, to obtain a first projection image; then the light source 1 is controlled to project a second projection pattern onto the object to be tested 2, and controls the image acquisition device 3 to capture the image of the object to be tested 2, to obtain a second projection image, until a preset number of projection patterns are projected, to obtain a preset number of projection images. In a possible implementation, the object to be tested 2 may also be placed on a translation stage, and the image acquisition device 3 is controlled to capture the image of the object to be tested 2 while the object to be tested 2 moves through the translation stage.

[0143] In another possible implementation, the pattern may be projected in a coaxial reflection manner. Figure 2b The diagram is a schematic diagram of projecting a pattern by coaxial reflection, wherein the lens optical axis of the image acquisition device 3 is perpendicular to the optical axis of the light source 1, the surface of the object 2 to be measured faces the image acquisition device 3, a beam splitter prism 4 is arranged between the image acquisition device 3 and the object 2 to be measured, and the beam splitter prism 4 vertically intersects the surface of the image acquisition device 3 and the object 2 to be measured, and the pattern projected by the light source 1 can be projected onto the surface of the object 2 to be measured through the beam splitter prism 4. In another possible implementation, for a transparent object to be measured, a transmission method can also be used to project the pattern. For example, Figure 2c The figure shows a schematic diagram of transmission projection, in which the light source 1 and the image acquisition device 3 are coaxially arranged, a motion platform 5 is arranged between the light source 1 and the image acquisition device 3, the object to be measured 2 is placed on the motion platform 5, and the pattern projected by the light source 1 is directly projected onto the object to be measured 2.

[0144] In order to accurately establish the surface information of the object to be measured, in a possible implementation, the same type of projection pattern usually includes multiple sub-patterns. In the above step S10, the light source is controlled to project at least one target type of projection pattern according to the surface properties of the object to be measured, which actually controls the light source to project each sub-pattern in the projection pattern of at least one target type to the object to be measured in turn.

[0145] When the projection pattern type is sine fringes or cosine fringes, each sub-pattern included in the projection pattern of the same type is actually a sine fringes or cosine fringes with different phases. Figure 3 The figure shows a first example of a sinusoidal fringe projection pattern provided in an embodiment of the present application. Figure 3 (1), (2), (3), and (4) are sub-patterns when the projected pattern is a sinusoidal stripe. The stripes in the pattern are vertical stripes, and there is a phase difference between the sub-patterns.

[0146] The pattern of sinusoidal stripes can also be transverse stripes, such as Figure 4 The figure shows a second example of a sinusoidal fringe projection pattern provided in an embodiment of the present application. Figure 4 (1), (2), (3), and (4) are sub-patterns when the projected pattern is a sinusoidal stripe, and there is a phase difference between the sub-patterns.

[0147] It can be understood that the waveforms of cosine fringes and sine fringes are both periodic. When the periods are the same, the only difference is the phase. Therefore, Figure 3 and Figure 4 The sub-patterns shown in can also be regarded as sub-patterns of cosine fringes. It can be understood that Figure 3 and Figure 4 The figure is only a schematic diagram of the sine fringe projection pattern / cosine fringe projection pattern, and does not mean that there are actually different styles of stripes in the sine fringe projection pattern / cosine fringe projection pattern. Figure 3 , Figure 4 The different styles of stripes in the image are used to indicate the brightness of the area where each stripe is located. Figure 3 and Figure 4 In the figure, an area filled with both horizontal and vertical stripes indicates that the brightness of the area is 1, an area filled only with horizontal stripes indicates that the brightness of the area is 2, an area filled with both left-low-right-high stripes and left-high-right-low stripes indicates that the brightness of the area is 3, an area filled only with left-high-right-low stripes indicates that the brightness of the area is 4, an area filled only with left-low-right-high stripes indicates that the brightness of the area is 5, an area without filled stripes indicates that the brightness of the area is 6, brightness 1<brightness 2<brightness 3<brightness 4<brightness 5<brightness 6.

[0148] Since the binary stripes present a simple black and white alternating or binary coded pattern, when the projection pattern is a binary stripe, the same type of projection pattern includes two sub-patterns, for example, Figure 5 The following is an example of a binary stripe. Figure 5 The difference between the sub-pattern (1) and the sub-pattern (2) is only the starting color of the stripes.

[0149] Sinusoidal fringes and pre-fringe can be generated by interference, diffraction, etc., or by algorithmic simulation to generate sinusoidal fringes or cosine fringes with phase distribution.

[0150] In a possible implementation, the sine stripes or cosine stripes may also be a projected sine stripe pattern, and by moving the object to be tested, sine stripes with different phases are displayed on the object to be tested. For ease of description, the pattern projection method with angled projection is taken as an example to illustrate the pattern projection process and the image acquisition process.

[0151] For example, Figure 6 Shown Figure 2a The first principle schematic diagram of the defect detection method of the embodiment shown in the figure, the illumination range of the light source 1 is shown in the black dotted line frame, and the light source 1 is controlled to display Figure 4 In the sinusoidal fringe pattern shown in (1), when the object 2 is at position A, the object 2 ( Figure 7 The dotted box in the figure shows the position of the projected sinusoidal fringe pattern. Figure 7 As shown in (A), the projection image acquired by the image acquisition device is as follows Figure 8 As shown in (A), when the object 2 moves to position B, the object 2 ( Figure 7 The dotted box in the figure shows the position of the projected sinusoidal fringe pattern. Figure 7 As shown in (B), the projection image acquired by the image acquisition device is as follows Figure 8 As shown in (B), Figure 8 (A) and (B) in the figure also show sinusoidal fringes with different phases.

[0152] The binary stripes can be directly generated by digital image processing software, or can be a binary-coded pattern projected by a physical projection device. The bright block pattern only has bright blocks and no other elements. Depending on the shape of the bright blocks, they can be round bright blocks, rectangular bright blocks, etc., or bright blocks with different light source characteristics.

[0153] When using the photometric stereo method for defect detection, multiple point light sources are usually used to illuminate the object to be tested in a time-sharing manner. The image acquisition device is controlled to capture the object to be tested when each light source is illuminated to obtain multiple images. The photometric stereo algorithm is then used to process the captured images to reconstruct the three-dimensional morphology, reflectivity, etc. of the object surface. The reconstructed images are then analyzed to identify defects on the object surface.

[0154] It can be seen that the core of the photometric stereo method lies in illumination from different directions. In order to achieve illumination from different directions, in a possible implementation, multiple preset projection areas can be set on the light source, such as Fig. 9 Shown Figure 2a The second principle diagram of the defect detection method of the embodiment shown. There are multiple light source beads on the light source 1. It can be understood that the dots of different styles in the light source 1 are only used to represent different projection areas. The dots of the same style represent the beads in the same projection area. The style of each dot does not represent the actual style of the lamp bead. Fig. 9 All the lamp beads of the middle light source 1 are of the same type. For example, the area where the black dots are located represents the projection area 1, the area where the white dots are located represents the projection area 2, and the area where the dots with a cross pattern are located represents the projection area 3.

[0155] like Fig.10 Shown Fig. 9 The equivalent schematic diagram of Fig. 9 Each projection area in the light source 1 shown can be approximately expressed as Fig.10 From the independent light sources in the image (shown by black dots, white dots, and dots with cross patterns), it can be seen that the orientations of the projection areas relative to the object to be tested 2 are different. When using the photometric stereo method for defect detection, different areas of the light source 1 are controlled to project the projection patterns corresponding to the photometric stereo method in a time-sharing manner to achieve illumination in different directions.

[0156] When two or more types of projection patterns are used to perform defect detection on an object to be detected, in order to more clearly illustrate the projection process of each type of pattern by the light source in the embodiment of the present application, in a specific embodiment, Fig. 9 As shown, assuming that photometric stereo method and phase deflection method are used to perform defect detection on the object 2 to be tested, the projection area where the lamp beads represented by black dots are located is controlled to project the projection pattern corresponding to the phase deflection method, and the projection area where the lamp beads represented by black dots are located, the projection area where the lamp beads represented by white dots are located, and the projection area where the color lamp beads represented by cross-patterned dots are located are controlled to project the projection pattern corresponding to the photometric stereo method.

[0157] During the image acquisition process, the camera can generate and output a projected image each time it photographs the object to be measured. In order to enrich the image details of the projected image acquired by the camera, the camera can also expose an image frame each time a sub-pattern is projected. After all sub-patterns are projected, the camera generates an original image including a preset number of image frames.

[0158] Based on this, in a possible implementation manner, Fig.11 The second schematic diagram of the defect detection method provided by the embodiment of the present application is shown. The above step S10 controls the camera to collect the projection image corresponding to the projection pattern of each target type, including:

[0159] Step S101, controlling the camera to collect a preset number of image frames to obtain an original image;

[0160] Wherein, the object to be measured in each image frame is projected with a different projection pattern;

[0161] Step S102: split the original image according to the type of projection pattern to obtain projection images corresponding to the projection patterns of each target type.

[0162] In this embodiment, the original image is split according to the type of projection pattern to obtain projection images corresponding to the projection patterns of each target type, which may refer to splitting the projection image according to the target type of the projection pattern, splitting the image frames containing sub-patterns belonging to the same target type in the same projection image, and obtaining projection images corresponding to the projection patterns of each target type. Alternatively, the projection image may be split according to the projection sub-pattern to obtain a preset number of projection sub-images, and then splicing the projection sub-images corresponding to the sub-patterns belonging to the same target type into one projection image, and obtaining projection images corresponding to the projection patterns of each target type.

[0163] For example, if there are three types of projection patterns, the first type of projection patterns includes stripes 11, stripes 12, and stripes 13, the second type of projection patterns includes stripes 21, stripes 22, and stripes 23, and the third type of projection patterns includes stripes 31, stripes 32, and stripes 33, then after the sub-patterns in the three types of projection patterns are cross-projected, the original image acquired by the image acquisition device is as follows: Fig.12a shown.

[0164] If the projection pattern is divided according to the target type, it is as follows Figure 12b As shown, the splitting results in a projection image including stripes 11, stripes 12, and stripes 13, a projection image including stripes 21, stripes 22, and stripes 23, and a projection image including stripes 31, stripes 32, and stripes 33.

[0165] If the image is split according to the projection sub-pattern, a preset number of projection sub-images containing different sub-patterns are obtained. Fig.12c For example, 9 projection sub-images are obtained, and then the projection sub-images containing stripes 11, stripes 12, and stripes 13 of the same target type are spliced ​​into one projection image, the projection sub-images containing stripes 21, stripes 22, and stripes 23 are spliced ​​into one projection image, and the projection sub-images containing stripes 31, stripes 32, and stripes 33 are spliced ​​into one projection image, such as Figure 12b As shown, a projection image corresponding to the projection pattern of each target type is obtained.

[0166] Based on the above, it can be known that there are actually multiple sub-patterns of different types of projection patterns. It can be understood that when performing defective parts testing, the number of sub-patterns of different types of projection patterns and the projection order are pre-set. For example, assuming that there are two types of projection patterns, projection pattern A includes 4 sub-patterns A1, A2, A3, and A4, and projection pattern B includes 3 sub-patterns B1, B2, and B3, then when controlling the light source 1 for projection, sub-patterns A1, A2, A3, and A4 may be projected first, and then sub-patterns B1, B2, and B3 may be projected; sub-patterns B1, B2, and B3 may be projected first, and then sub-patterns A1, A2, A3, and A4 may be projected; or sub-patterns of projection pattern A may be cross-projected with sub-patterns of projection pattern B, for example, each sub-pattern may be projected in the order of A1, B1, A2, B2, A3, B3, and A4. All of these are possible.

[0167] Based on this, in order to implement the placement of each sub-pattern in a preset order, each sub-pattern needs to be numbered. In a possible implementation, the defect detection method further includes:

[0168] Each sub-pattern is numbered based on the type of projection pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of types of projection patterns, and N is the total number of projection patterns of the same type.

[0169] In a possible implementation manner, if the sub-patterns of each type of projection pattern are projected in a cross-projection manner, then Fig.13a As shown, the above step S10 controls the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controls the camera to collect projection images corresponding to the projection patterns of each target type, which may specifically include the following steps:

[0170] Step S01: P i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0171] Then determine whether i is less than M. If i is less than M, execute step S02; if i is equal to M, execute step S03;

[0172] Step S02: P i+1，j As the new current sub-pattern, and return to step S01 until i is equal to M;

[0173] Step S03: i，j+1 As the new current sub-pattern, the process returns to step S01 until j is equal to N.

[0174] In another possible implementation, if the projection is performed according to the type of projection pattern, then Fig.13b As shown, the above step S10 controls the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controls the camera to collect projection images corresponding to the projection patterns of each target type, which may specifically include the following steps:

[0175] Step S001: P i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0176] Then determine whether j is less than N. If j is less than N, execute step S002. If j is equal to N, execute step S003.

[0177] Step S002: P i，j+1 As the new current sub-pattern, and return to step S001 until j is equal to N;

[0178] Step S003: i+1，j As the new current sub-pattern, the process returns to step S001 until i is equal to M.

[0179] It can be understood that in step S02 and step S003, after determining the new current sub-pattern, P i+1，j When P is used as the new current sub-pattern, i，j When the size of i and j is , i = i + 1 at this time, and in step S03 and step S002, after determining the new current sub-pattern, P i，j+1 When P is used as the new current sub-pattern, i，j When the size of i and j is large, j = j+1.

[0180] Exemplarily, the current sub-pattern in step S001 is P 1，1 , if N is 3 and M is 4, then 1<3, and step S002 is executed to set P 1，2 As the new current projected image, since 2<3, continue to execute step S002 and set P 1，3 As the new current projected image, since j is equal to 3 at this time, step S002 is executed to convert P 2，1 as the new current projected image, and so on, until i=M and N equals 4.

[0181] Understandably, P i，j This is just a numbering method for the projected sub-patterns. Other numbering methods may also be used to number the sub-patterns, such as (i, j), ij. This is all possible and the present application does not limit this.

[0182] In order to more clearly illustrate the projection method of each sub-pattern, a description is given below in conjunction with a specific embodiment.

[0183] In a specific embodiment, assuming that there are three types of projection patterns and there are M sub-patterns in each type of projection pattern, the sub-patterns in the first type of projection pattern are numbered 11, 12, 13, 14, 15...1M; the sub-patterns in the second type of projection pattern are numbered 21, 22, 23, 24, 25...2M; and the sub-patterns in the third type of projection pattern are numbered 31, 32, 33, 34, 35...3M.

[0184] If adopted Fig.13a The sub-patterns are cross-projected in the manner shown, the light source is controlled to project the sub-patterns in the order of 11, 21, 31, 12, 22, 32, 13, 23, 33 ... 1M, 2M, 3M in a time-sharing manner, and the image acquisition device is controlled to shoot the object to be measured when projecting each sub-pattern.

[0185] If adopted Fig.13b The sub-patterns are cross-projected in the manner shown, and the light source is controlled to project the sub-patterns in the order of 11, 12, 13, 14, 15 ... 1M, 21, 22, 23, 24, 25 ... 2M, 31, 32, 33, 34, 35 ... 3M in a time-sharing manner, and the image acquisition device is controlled to shoot the object to be measured when projecting each sub-pattern.

[0186] It can be understood that the camera in the embodiment of the present application can be a line array camera or an area array camera, and the embodiment of the present application is not limited to this.

[0187] When the image acquisition device is a linear array camera, the number of pixel rows that the linear array camera can scan at a time is limited. For example, the linear array camera can only scan x rows of pixels each time. Then the number of rows of the original image generated by the linear array camera when executing the above step S101 should be xy, where y is the number of sub-patterns. For example, there are 3 types of projected patterns, each of which has 4, 6, and 8 sub-patterns. The acquisition height of the linear array camera is 5000 rows of pixels. Then, the same position on the surface of the object to be measured needs to be acquired 4+6+8=18 times, and the total number of rows of the original image is (4+6+8) 5000=90000 rows.

[0188] The pattern processing process is described below:

[0189] It is understandable that different types of projection patterns usually use different image processing algorithms when performing image processing. For example, projection patterns of sine stripes or cosine stripes are suitable for phase deflection method, and projection patterns of binary stripes are suitable for simple binary image analysis method.

[0190] In order to use a suitable image processing algorithm to process the projection images corresponding to the projection patterns of each target type captured by the camera, it is also necessary to establish the correspondence between the types of projection patterns and the image processing algorithms in advance. In a possible implementation, the correspondence between the types of projection patterns and the image processing algorithms is configured in the following manner:

[0191] In response to the relationship configuration instruction, determine the image processing algorithm indicated by the relationship configuration instruction and the type of the projection pattern indicated by the relationship configuration instruction; and establish a corresponding relationship between the image processing algorithm indicated by the configuration establishment instruction and the projection pattern indicated by the relationship configuration instruction.

[0192] In order to facilitate the subsequent defect detection using the defect detection algorithm corresponding to the projection pattern type and improve the defect detection efficiency, in a possible implementation, after controlling the camera to collect projection images corresponding to the projection patterns of each target type in the above step S10, each projection image can be divided into multiple image groups of target types according to the type of the projection image, so that for each image group, the projection images in each image group are processed using the image processing algorithm corresponding to the target type of each image group. Based on this, Fig.14 As shown, the above step S20 may specifically include the following steps:

[0193] Step S201, dividing each projection image into a plurality of target type image groups according to the type of projection pattern;

[0194] The types of projection patterns in the projection images in the same image group are the same, and the types of projection patterns in the projection images in different image groups are different;

[0195] Step S202 : for each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

[0196] For example, assuming that among the 6 projection images captured sequentially by the camera, the projection patterns in the 1st, 3rd, and 5th projection images are the projection patterns of target type 1, and the projection patterns in the 2nd, 4th, and 6th projection images are the projection patterns of target type 2, then the 1st, 3rd, and 5th projection images are divided into the same image group 1, and the 2nd, 4th, and 6th projection images are divided into the same image group 2. Assuming that the image processing algorithm corresponding to target type 1 is image processing algorithm 1, and the image processing algorithm corresponding to target type 2 is image processing algorithm 2, then image processing algorithm 1 is used to process the images in image group 1, and image processing algorithm 2 is used to process the images in image group 2.

[0197] By adopting the method of the embodiment of the present application, each projection image is divided into different image groups according to the target type of the projected projection pattern. In this way, projection images of the same target type are in the same image group, and each projection image in the image group can be processed by using the image processing algorithm corresponding to the target type of the image group, thereby improving the processing efficiency of each projection image.

[0198] Finally, the defect detection process is explained:

[0199] Based on the above embodiments, it can be known that the surface properties of the object to be tested in the embodiments of the present application may include multiple types. Therefore, when the surface properties of the object to be tested include multiple types, different image processing algorithms need to be used to process the projection images to determine all defects on the surface to be tested. Specifically, each projection image is processed using the image processing algorithm corresponding to the projection pattern of each target type to obtain the first defect detection results respectively, and the first defect detection results are fused to obtain all defects of the object to be tested, that is, the target defect detection results.

[0200] Exemplarily, assume that the light source is controlled to project patterns corresponding to the photometric stereo method and the phase deflection method onto the object to be measured, and the corresponding projection images are collected. After the projection images are divided into image group A and image group B according to the target type of the projection pattern, the photometric stereo method is used to perform defect detection on image group A to obtain defect detection result A, and the phase deflection method is used to perform defect detection on image group B to obtain defect detection result B. Then, the detection result A and the detection result B are fused to obtain the target detection result of the object to be measured.

[0201] A second aspect of the present application provides a defect detection system, such as Fig.15 As shown, the system includes a light source 1501, a camera 1502, a first controller 1503 and a second controller 1504;

[0202] A light source 1501 is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0203] Camera 1502, used to collect projection images;

[0204] The first controller 1503 is used to control the light source to project a projection pattern of at least one target type according to the surface properties of the object to be measured, and to control the camera to collect projection images corresponding to the projection patterns of each target type;

[0205] The second controller 1504 is used to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result; and to fuse each of the first defect detection results to obtain a target defect detection result.

[0206] The first controller 1503 and the second controller 1504 may be the same controller or two independent and different controllers, which is not limited in this embodiment of the present application.

[0207] The surface properties of the object to be measured obtained by recognizing the first image captured by the camera can be obtained by the camera 1502 with recognition capability, or by the first controller 1503, or by the second controller 1504, and this embodiment of the application is not limited to this.

[0208] It is understandable that, in the case where the first controller 1503 and the second controller 1504 are two independent and different controllers, the first controller 1503 is also used to send the projection image captured by the camera 1502 to the second controller 1504 .

[0209] Through the defect detection system of the embodiment of the present application, since different types of projection patterns and their corresponding image processing algorithms can detect different types of defects, when performing defect detection on the object to be tested, no matter whether the surface properties of the object to be tested are one or more, it is only necessary to select the corresponding projection pattern according to the surface properties of the object to be tested, and control the camera to capture the corresponding projection image, and then use the image processing algorithm corresponding to the projection pattern to process the image, so as to obtain the defect detection result of the object to be tested, without formulating a corresponding detection plan according to the surface properties of the object to be tested. Defect detection can be performed quickly on the product to be tested, thereby improving the defect detection efficiency.

[0210] In a possible implementation manner, the second controller 1504 is specifically configured to:

[0211] According to the type of the projection pattern, each projection image is divided into multiple image groups of target types, wherein the type of the projection pattern in each projection image in the same image group is the same, and the type of the projection pattern in each projection image in different image groups is different;

[0212] For each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

[0213] In a possible implementation manner, the first controller 1503 is specifically configured to:

[0214] Obtaining a surface attribute selection instruction for the object to be measured, and controlling the light source to project a projection pattern of a target type corresponding to the surface attribute indicated by the surface attribute selection instruction; wherein the correspondence between the surface attribute indicated by the surface attribute selection instruction and the type of the projection pattern is pre-configured;

[0215] or,

[0216] In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, controlling the light source to project a projection pattern of at least one target type indicated by the pattern selection instruction;

[0217] or,

[0218] The surface properties of the object to be measured are obtained by identifying the first image captured by the camera, and the light source is controlled to project a projection pattern of a target type corresponding to the surface properties, wherein the correspondence between the surface properties and the type of the projection pattern is pre-configured.

[0219] In a possible implementation, the correspondence between the surface attributes and the types of projection patterns is pre-configured in the following manner:

[0220] In response to the relation establishment instruction, determining the surface attribute indicated by the relation establishment instruction and the type of the projection pattern indicated by the relation establishment instruction;

[0221] A correspondence between the surface attribute indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

[0222] In a possible implementation, the projection pattern of the same target type includes multiple sub-patterns, and different projection areas are provided on the light source, and each projection area has a different orientation relative to the object to be measured;

[0223] The image processing algorithm includes a photometric stereo method, and each sub-pattern included in the projection pattern corresponding to the photometric stereo method is projected onto the object to be measured from different projection areas.

[0224] In a possible implementation manner, the first controller 1503 is specifically configured to:

[0225] Controlling the camera to collect a preset number of image frames respectively to obtain original images, wherein the projection pattern projected on the object to be measured in each image frame is different;

[0226] The original image is split according to the type of projection pattern to obtain projection images corresponding to the projection patterns of each target type.

[0227] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the system further includes:

[0228] A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0229] The first controller 1503 is specifically used for:

[0230] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0231] If i is less than M, then P i+1，j As the new current sub-pattern, return to execute P i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0232] If i is equal to M, then P i，j+1 As the new current sub-pattern, return to execute P i，j The step of controlling the light source to project the current sub-pattern onto the object to be measured as the current sub-pattern, and controlling the camera to collect a projected image corresponding to the current sub-pattern.

[0233] In a possible implementation manner, the projection pattern of the same target type includes multiple sub-patterns, and the system further includes:

[0234] A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type;

[0235] The first controller 1503 is specifically used for:

[0236] P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0237] If j is less than N, then P i，j+1 As the new current sub-pattern, return to execute P i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern;

[0238] If j is equal to N, then P i+1，j As the new current sub-pattern, return to execute P i，j The step of controlling the light source to project the current sub-pattern onto the object to be measured as the current sub-pattern, and controlling the camera to collect a projected image corresponding to the current sub-pattern.

[0239] Corresponding to the first aspect, the third aspect of the embodiment of the present application provides a defect detection device, such as Fig.16 As shown, the device comprises:

[0240] The control module 1601 is used to control the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and control the camera to collect projection images corresponding to the projection patterns of each target type; wherein the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured;

[0241] A processing module 1602 is used to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result;

[0242] The fusion module 1603 is used to fuse the first defect detection results to obtain a target defect detection result.

[0243] Through the defect detection device of the embodiment of the present application, since different types of projection patterns and their corresponding image processing algorithms can detect different types of defects, when performing defect detection on the object to be tested, no matter whether the surface properties of the object to be tested are one or more, it is only necessary to select the corresponding projection pattern according to the surface properties of the object to be tested, and control the camera to capture the corresponding projection image, and then use the image processing algorithm corresponding to the projection pattern to process the image, so as to obtain the defect detection result of the object to be tested, without formulating a corresponding detection plan according to the surface properties of the object to be tested. Therefore, defect detection can be performed quickly on the product to be tested, thereby improving the defect detection efficiency.

[0244] A fourth aspect of the embodiments of the present application further provides an electronic device, such as Fig.17 As shown, including:

[0245] Memory 1701, used for storing computer programs;

[0246] The processor 1702 is used to implement any of the above-mentioned defect detection methods when executing the program stored in the memory 1701.

[0247] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0248] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0249] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned defect detection methods are implemented.

[0250] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any defect detection method in the above embodiments.

[0251] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by the computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0252] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0253] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for system and device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0254] The above are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the protection scope of the present application.

Claims

1. A defect detection method, characterized in that: The method comprises: Controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type; wherein the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured; Processing each of the projection images using an image processing algorithm corresponding to the projection pattern of each of the target types to obtain a first defect detection result; The first defect detection results are fused to obtain a target defect detection result.

2. The method according to claim 1, characterized in that The step of processing each of the projection images using an image processing algorithm corresponding to the projection pattern of each of the target types to obtain a first defect detection result includes: According to the type of the projection pattern, each projection image is divided into a plurality of image groups of target types, wherein the type of the projection pattern in each projection image in the same image group is the same, and the type of the projection pattern in each projection image in different image groups is different; For each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

3. The method according to claim 1, characterized in that The controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured comprises: Obtaining a surface attribute selection instruction for the object to be measured, and controlling a light source to project a projection pattern of a target type corresponding to the surface attribute indicated by the surface attribute selection instruction; wherein the correspondence between the surface attribute indicated by the surface attribute selection instruction and the type of the projection pattern is pre-configured; or, In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, controlling the light source to project a projection pattern of at least one target type indicated by the pattern selection instruction; or, Acquire the surface properties of the object to be measured obtained by identifying the first image captured by the camera, and control the light source to project a projection pattern of a target type corresponding to the surface properties, wherein the correspondence between the surface properties and the type of projection pattern is pre-configured.

4. The method according to claim 3, characterized in that The correspondence between the surface attributes and the type of the projection pattern is pre-configured in the following manner: In response to the relationship establishment instruction, determining the surface attribute indicated by the relationship establishment instruction and the type of the projection pattern indicated by the relationship establishment instruction; A correspondence between the surface attribute indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

5. The method according to claim 1, characterized in that The projection pattern of the same target type includes a plurality of sub-patterns, and different projection areas are arranged on the light source, and the positions of the projection areas relative to the object to be measured are different; The image processing algorithm includes a photometric stereo method, and each sub-pattern included in a projection pattern corresponding to the photometric stereo method is projected onto the object to be measured from different projection areas.

6. The method according to claim 1, characterized in that The projection pattern of the same target type includes multiple sub-patterns, and the control camera collects projection images corresponding to the projection patterns of each target type, including: Controlling the camera to collect a preset number of image frames respectively to obtain original images, wherein the projection pattern projected on the object to be measured in each image frame is different; The original image is split according to the types of the projection patterns to obtain projection images corresponding to the projection patterns of each target type.

7. The method according to claim 1, characterized in that The projection pattern of the same target type includes multiple sub-patterns, and the method further includes: Each sub-pattern is numbered based on the type of projection pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type; The controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type, comprises: P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projected image corresponding to the current sub-pattern; If i is less than M, then P i+1，j As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern; If i is equal to M, then P i，j+1 As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

8. The method according to claim 1, characterized in that The projection pattern of the same target type includes multiple sub-patterns, and the method further includes: Each sub-pattern is numbered based on the type of projection pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type; The controlling the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and controlling the camera to collect projection images corresponding to the projection patterns of each target type, comprises: P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern; If j is less than N, then P i，j+1 As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern; If j is equal to N, then P i+1，j As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

9. A defect detection system, characterized in that: The defect detection system includes a light source, a camera, a first controller and a second controller; The light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured; The camera is used to collect the projected image; The first controller is used to control the light source to project a projection pattern of at least one target type according to the surface properties of the object to be measured, and to control the camera to collect projection images corresponding to the projection patterns of each target type; The second controller is used to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result; and to fuse each of the first defect detection results to obtain a target defect detection result.

10. The system according to claim 9, characterized in that The second controller is specifically used for: According to the type of the projection pattern, each projection image is divided into a plurality of image groups of target types, wherein the type of the projection pattern in each projection image in the same image group is the same, and the type of the projection pattern in each projection image in different image groups is different; For each image group, the projection images in the image group are processed using an image processing algorithm corresponding to the target type of the image group to obtain a first defect detection result.

11. The system according to claim 9, characterized in that The first controller is specifically used for: Obtaining a surface attribute selection instruction for the object to be measured, and controlling a light source to project a projection pattern of a target type corresponding to the surface attribute indicated by the surface attribute selection instruction; wherein the correspondence between the surface attribute indicated by the surface attribute selection instruction and the type of the projection pattern is pre-configured; or, In response to a pattern selection instruction for the object to be measured input according to the surface properties of the object to be measured, controlling the light source to project a projection pattern of at least one target type indicated by the pattern selection instruction; or, Acquire the surface properties of the object to be measured obtained by identifying the first image captured by the camera, and control the light source to project a projection pattern of a target type corresponding to the surface properties, wherein the correspondence between the surface properties and the type of projection pattern is pre-configured.

12. The system according to claim 11, characterized in that The correspondence between the surface attributes and the type of the projection pattern is pre-configured in the following manner: In response to the relationship establishment instruction, determining the surface attribute indicated by the relationship establishment instruction and the type of the projection pattern indicated by the relationship establishment instruction; A correspondence between the surface attribute indicated by the relationship establishment instruction and the projection pattern indicated by the relationship establishment instruction is established.

13. The system according to claim 9, characterized in that The projection pattern of the same target type includes a plurality of sub-patterns, and different projection areas are arranged on the light source, and the positions of the projection areas relative to the object to be measured are different; The image processing algorithm includes a photometric stereo method, and each sub-pattern included in a projection pattern corresponding to the photometric stereo method is projected onto the object to be measured from different projection areas.

14. The system according to claim 9, characterized in that The first controller is specifically used for: Controlling the camera to collect a preset number of image frames respectively to obtain original images, wherein the projection pattern projected on the object to be measured in each image frame is different; The original image is split according to the types of the projection patterns to obtain projection images corresponding to the projection patterns of each target type.

15. The system according to claim 9, characterized in that The projection pattern of the same target type includes multiple sub-patterns, and the system further includes: A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type; The first controller is specifically used for: P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern; If i is less than M, then P i+1，j As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern; If i is equal to M, then P i，j+1 As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

16. The system according to claim 9, characterized in that The projection pattern of the same target type includes multiple sub-patterns, and the system further includes: A numbering module is used to number each sub-pattern based on the type of the projected pattern, denoted as P i，j , represents the jth sub-pattern of the i-th category, where i=1, 2...M, j=1, 2...N, M is the total number of projected pattern types, and N is the total number of sub-patterns of the same type; The first controller is specifically used for: P i，j As the current sub-pattern, initially i=1, j=1, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to capture the projection image corresponding to the current sub-pattern; If j is less than N, then P i，j+1 As the new current sub-pattern, return to execute the i，j as a current sub-pattern, controlling a light source to project the current sub-pattern onto an object to be measured, and controlling a camera to capture a projected image corresponding to the current sub-pattern; If j is equal to N, then P i+1，j As the new current sub-pattern, return to execute the i，j As the current sub-pattern, control the light source to project the current sub-pattern onto the object to be measured, and control the camera to collect a projected image corresponding to the current sub-pattern.

17. A defect detection device, characterized in that: The device comprises: A control module, used to control the light source to project at least one target type of projection pattern according to the surface properties of the object to be measured, and to control the camera to collect projection images corresponding to the projection patterns of each target type; wherein the light source is used to project at least two different types of projection patterns corresponding to different surface properties of the object to be measured; A processing module, configured to process each of the projection images using an image processing algorithm corresponding to the projection pattern of each target type to obtain a first defect detection result; A fusion module is used to fuse the first defect detection results to obtain a target defect detection result.

Images