Method, system and equipment for detecting surface defects by photometric stereo method

Abstract

The invention relates to the technical field of semiconductor detection, in particular to a photometric stereo method surface defect detection method, system and equipment. A corresponding partition control image acquisition device is configured to acquire a plurality of sub-images related to a to-be-detected chip, and the plurality of sub-images are subjected to stereo reconstruction to obtain a 2.5 D image related to the to-be-detected chip; and identifying the 2.5 D image through a target detection model with a coordinate attention mechanism, and identifying an object with a depth value.

Description

Technical Field

[0001] The present application relates to the field of semiconductor detection technology, and is a photometric stereo surface defect detection method, system and equipment. Background Art

[0002] Due to the actual process problems in the packaging process, the actual material condition of the packaged chip is poor, and the surface visual defect detection will lead to inaccurate recognition of key features due to process reasons. For example, when the exact position of the probe mark on the packaged chip is tested, particles or surface oxidation may be imaged in 2D conditions with the same grayscale as the probe mark, resulting in failure to inspect. Summary of the invention

[0003] In order to solve the above problems caused by objective processes, the present application provides a stereoscopic surface defect detection method, system, equipment and storage medium, which obtains the depth values ​​of different objects in the image by stereoscopically reconstructing the collected image, and performs target detection based on the depth value.

[0004] In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0005] In a first aspect, a stereoscopic surface defect detection method is provided, the method comprising: stereoscopically reconstructing a plurality of planar images acquired in light source scenes with different directions to obtain a non-planar image of an object to be detected; the light source scene is a rectangular ring light, and the light source in each direction can be controlled individually; performing regional segmentation on the non-planar image, and determining the depth information of the segmented area, and determining the area that meets the depth information as the target area.

[0006] In some specific implementations, stereoscopic reconstruction is performed on the multiple plane images, including: obtaining light intensities corresponding to the multiple plane images, and determining reflectivities and unit normal vectors corresponding to the multiple plane images based on the light intensities; obtaining first surface gradients and second surface gradients corresponding to the multiple plane images according to the reflectivities and the unit normal vectors; the first surface gradients and the second surface gradients are reconstruction elements of the plane images.

[0007] In some specific implementations, the non-planar image is a depth map including depth values, and obtaining the non-planar image of the object to be detected includes: obtaining the depth map by LLT factorization based on the first surface gradient, the second surface gradient and a sparse matrix.

[0008] In some specific implementations, the depth map is segmented into regions through a target detection model; the target detection model includes a backbone network, a neck structure, and a head structure, the backbone network is used to extract the features of the depth to obtain a feature map, the neck structure incorporates a coordinate attention mechanism to combine position information and channel attention, and is used to differentially encode the position information of each pixel in the feature map and determine the attention weight of each pixel.

[0009] In some specific implementations, the backbone network includes a FasterNet Block structure and a Focus structure; the Focus structure is composed of a convolution block, which is then convolved through a filter, and the final result outputs a convolution kernel, and the output effect reduces the width and height of the image by half.

[0010] In some specific implementations, the differential encoding process includes: initially encoding the feature map through two one-dimensional encoding processes, then performing global average pooling along the width and height dimensions to split it into different dimensions to obtain the corresponding first feature map and second feature map, and obtaining the coordinate attention weight of each channel of the first feature map and the second feature map through a coordinate attention mechanism, and obtaining a connected feature map based on the weights from the first feature map and the second feature map.

[0011] In some specific implementations, a batch normalization layer and a nonlinear activation layer are provided in the head structure, which are used to split the connection feature map in width and height dimensions, retain the feature map in the height dimension as the target feature map, and determine the target area based on the depth distribution on the target feature map.

[0012] In a second aspect, a stereoscopic surface defect detection system is provided, the system comprising an image acquisition device and a processing device, the image acquisition device comprising an annular light source arranged above the object to be detected and whose center position is coaxial with the center position of the object to be detected, the annular light source comprising four sub-light sources, each of the sub-light sources being used to separately acquire a plane image within a corresponding light source angle range; the processing device being used to acquire the plane image and execute the method described in any one of claims 1 to 7.

[0013] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the computer program.

[0014] According to a fourth aspect, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0015] In the technical solution provided in the embodiment of the present application, the image acquisition device is controlled by configuring corresponding partitions to collect multiple sub-images about the chip to be detected, and the multiple sub-images are stereoscopically reconstructed to obtain a 2.5D image of the chip to be detected. The 2.5D image is recognized through a target detection model with a coordinate attention mechanism to identify objects with depth values. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] The methods, systems and / or programs in the accompanying drawings will be further described according to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example numbers represent similar mechanisms in the various views of the accompanying drawings.

[0018] Figure 1 It is a packaged chip image in the background technology.

[0019] Figure 2 It is a schematic diagram of the structure of the defect detection system provided in an embodiment of the present application.

[0020] Figure 3 It is a schematic flow chart of the three-dimensional surface defect detection method provided in an embodiment of the present application.

[0021] Figure 4 It is a schematic diagram of the structure of the processing device provided in an embodiment of the present application.

[0022] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0023] Figure 6 It is a schematic diagram of the readable medium structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0025] In the following detailed description, numerous specific details are set forth by way of example in order to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that the present application may be practiced without these details. In other cases, well-known methods, procedures, systems, compositions and / or circuits have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present application.

[0026] Flowcharts are used in the present application to illustrate the execution process performed by the system according to the embodiment of the present application. It should be clearly understood that the execution process of the flowchart may not be performed in order. On the contrary, these execution processes may be performed in reverse order or simultaneously. In addition, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0027] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0028] (1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0029] (2) Based on is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0030] See also Figure 1 , is the image of the packaged chip, which is the detection object of this embodiment. This image is a plane image and is acquired by optical equipment. Figure 1 It can be seen that due to process reasons, some poor process rectangular pads and PM hole marks cannot be accurately identified and distinguished for packaged chips, resulting in over-inspection at the detection end.

[0031] Therefore, in order to accurately distinguish and identify the pad contour and the PM needle mark and achieve the technical purpose of accurate detection, it is necessary to accurately identify the objects in this scene. The applicant analyzed the process of the packaged chip with the above problem and found that there is a height difference between the base of the pad and the die, and there is also a height difference between the PM puncture marks on the pad; therefore, the object can be identified by determining the height difference between different objects. However, the image captured by the optical device of the prior art is a planar image, and the height value of the corresponding object cannot be determined on this image. Therefore, in order to achieve the above technical effect, a defect detection system is provided in this embodiment, which collects a non-planar image with height information of the packaged chip through an optical device different from the prior art, and reconstructs the image to obtain a depth map, and finally uses the depth map as the recognition base image to identify different objects, thereby solving the technical problems existing in the prior art.

[0032] Specifically, the system 20 may refer to 2, which includes an image acquisition device 21 and a processing device 22, wherein the image acquisition device is used to obtain a non-planar image of the packaged chip 10, and the processing device is used to reconstruct and identify the non-planar image, and identify objects with different height values.

[0033] Among them, this image acquisition device is used to realize subsequent image reconstruction, and the reconstructed image has an expression of depth information, which is a lighting arrangement relative to the four sides of the packaged chip to be detected. This kind of lighting arrangement can realize the depth expression of the packaged chip to be detected. Specifically, this image acquisition device should include an annular light source arranged above the packaged chip to be detected, and its center position is coaxial with the center position of the object to be detected. It is worth noting that the so-called center position coaxial refers to the center position of the annular light source, and does not refer to the center position of the illuminating surface. For this annular light source, a rectangular light source is preferably selected, and the shape configuration of this light source should be the same as the shape of the packaged chip to be detected.

[0034] Four sub-light sources 211 with independent image acquisition are arranged on the annular light source, and the light emitting surface of each sub-light source is set corresponding to an edge position of the packaged chip to be inspected, and the light emitting surface of each sub-light source has a collection angle relative to the edge position. In this way, the four sub-light sources can be used for lighting, and the sensor 212 can be used to obtain highly rendered images of different areas on the packaged chip to be inspected.

[0035] The sensor obtains four images of different areas on the packaged chip to be detected, and the four images can fully express the entire area on the packaged chip to be detected. The four images are transmitted to the processing device, and the processing device executes the corresponding detection method.

[0036] In other embodiments, the ring-shaped light source may include other numbers of sub-images, but it should be noted that the number of the sub-light source and the corresponding sub-images should be at least four.

[0037] For the processing device to implement the three-dimensional surface defect detection method in this embodiment, please refer to Figure 3 , including the following steps:

[0038] Step S31: stereoscopically reconstruct multiple regional images acquired in scenes with light sources in different directions to obtain a non-planar image of the object to be detected.

[0039] In this embodiment, for the images of the chip to be detected at four positions on the chip to be detected acquired in the above system, that is, the four regional images contain images of partial objects on partial regions of the chip to be detected. In order to obtain an image of the entire chip to be detected, it is necessary to fuse and reconstruct the above four sub-images to obtain a complete non-planar image of the chip to be detected.

[0040] Among them, the non-planar image is a 2.5D image. As for how to reconstruct the image, the photometric stereo reconstruction method is adopted in this embodiment. The photometric stereo method can obtain the surface gradient, pixel reflectivity and normal vector of the object to be tested, and then obtain the depth information of the object to be tested, so as to reconstruct the depth map of the chip to be tested.

[0041] Specifically, the intensity of diffuse reflection light based on the Lambertian diffuse reflection model is calculated as follows: I = ρLN, where I is the brightness of a single pixel in the image of the object being measured, and ρ is the surface reflectivity of the object being measured. is the light source unit direction vector, N=(N x ,N y ,N z ) T is the unit normal vector of the surface of the object being measured.

[0042] The brightness of the four sub-images obtained by the sensor under different angles of light can be determined based on the following formula: I = (I 1 ,I 2 ,I 3 ,I 4 ). That is, the above two expressions of brightness can be expanded into a linear equation group in the x, y, and z directions related to N:

[0043] Convert the above formula into matrix form as follows:

[0044] Where N and L are both unit normal vectors, so the reflectivity of the object under test can be calculated based on the least squares method to obtain the reflectivity and unit normal vector: ρ = || L -1 I||, According to the above formula, the surface gradient values ​​p and q of the chip to be tested in the x and y directions can be calculated:

[0045] Then, the surface gradient, pixel points, reflectivity and normal vector of the chip to be tested can be obtained through the processing of the above formula.

[0046] And, the height value is expressed in accordance with the following formula: M·z=p*q, where z is the height value and M is a sparse matrix. The sparse matrix is ​​first constructed using the SparseMatrix class according to the image size of the above four sub-images collected, and then the height value is solved using LLT factorization.

[0047] Finally, a depth map containing three-dimensional information is obtained by substituting the above z value into the grayscale according to linear normalization. This depth map is a non-planar image of the object to be detected.

[0048] Step S32: segment the non-planar image into objects, determine the depth information of the segmented objects, and determine the objects that meet the depth information as target objects.

[0049] In this embodiment, a target detection model for object recognition is implemented for the depth map obtained in step S31. Among them, the model infrastructure in this implementation example adopts the structure of YOLOv5, but it is different from the prior art in that the depth map obtained in step S31 has a small amount of information and is not semantically rich. If the existing YOLOv5 model is used, the depth object can be determined. Therefore, in the model in this embodiment, by adding a coordinate attention mechanism, the attention to position information when processing spatial information is improved.

[0050] Specifically, the target detection model of this embodiment includes a backbone network, a neck structure and a head structure. The backbone network is used to extract the features of the depth map to obtain a feature map, the neck structure incorporates a coordinate attention mechanism to combine the position information and channel attention, and is used to differentially encode the position information of each pixel in the feature map and determine the attention weight of each pixel; the head structure is used to output the predicted category.

[0051] The backbone network includes a FasterNet Block structure and a Focus structure; the Focus structure is composed of a convolution block, which is then convolved through a filter, and the final result is a convolution kernel output, and the output effect halves the width and height of the image. The FasterNet Block structure is configured with a local convolution layer and two 1×1 convolution blocks, and a batch normalization layer and a rectified linear unit activation function are configured between the two convolution blocks. The convolution block is followed by a residual structure and an identity mapping merge operation as the final result. For the residual structure and identity mapping merge operation, any solution in the prior art can be used.

[0052] In this embodiment, the ratio of the number of channels in the local convolution to the traditional convolution is 1 / 4, so the number of floating-point operations per second of the local convolution is only 1 / 16 of the original ordinary convolution. The local convolution layer can better handle the situation where the input contains occlusion by correcting the weight of the occluded area in the convolution kernel. In this embodiment, by configuring the FasterNet Block structure, problems such as missing data and incomplete images can be effectively handled, and the speed and performance of the model can be improved while maintaining a high accuracy. It can retain rich information from the lower-level defect location feature layer for subsequent feature fusion.

[0053] The Focus structure consists of a convolution block, which is then convolved through a filter. The final result outputs a convolution kernel, and the output effect halves the width and height of the image.

[0054] The neck structure is used to receive the processed feature map and perform difference encoding. A channel and spatial attention module is provided in the neck structure, wherein the channel and spatial attention module includes a channel attention submodule and a spatial attention submodule. The difference encoding process initially encodes the feature map through two one-dimensional encoding processes, and then performs global average pooling along the width and height dimensions to split the feature map into different dimensions, thereby obtaining the corresponding first feature map and second feature map.

[0055] After obtaining the feature maps along the width and height dimensions, they are connected. In the connection process, the corresponding coordinate attention weights are configured. Specifically, the first feature map and the second feature map are passed through the channel and spatial attention module to obtain the coordinate attention weights of each channel, and the first feature map and the second feature map are connected to obtain the connected feature map based on the weights.

[0056] Among them, for the processing process of the channel and spatial attention modules, first, the channel attention sub-module is configured with two operations, average pooling and maximum pooling, and then input into the shared multi-layer perception network. After passing through the shared network, addition and activation function processing are performed respectively to obtain the final channel attention focus information; the spatial attention sub-module first performs 1x1 convolution to reduce the dimension, and obtains the results of average pooling and maximum pooling respectively, and then splices them into a feature map, and then the convolution layer is used to learn the feature information, and finally the spatial focus position information is obtained through the activation function.

[0057] After obtaining the connected feature map, the batch normalization layer and the nonlinear activation layer are used to realize the splitting in the width and height dimensions, and the feature map in the height dimension is retained as the target feature map. The target feature map is identified through the head structure, and the target object is determined based on the depth distribution.

[0058] Among them, the target object in this embodiment is the probe mark. Since the probe mark has depth information, in this embodiment, the image is reconstructed into a 2.5D image to obtain a reconstructed image containing depth information; and by configuring the attention mechanism through the improved target detection model, it is possible to recognize objects with depth information and determine their accurate spatial coordinates.

[0059] Of course, it is worth noting that in other embodiments, the present solution is not only applicable to the recognition and detection of probe marks, but can also be applied to the detection of any object with depth information on a plane, including but not limited to target objects such as probe marks.

[0060] See also Figure 4 In this embodiment, a virtual structure is also provided for the processing device, and the processing device 22 includes the following modules:

[0061] The reconstruction module 221 is used to perform stereoscopic reconstruction on multiple regional images acquired in scenes with light sources in different directions to obtain a non-planar image of the object to be detected;

[0062] The segmentation module 222 is used to segment the non-planar image into objects, determine the depth information of the segmented objects, and determine the objects that meet the depth information as target objects.

[0063] Among them, the processing process of stereo reconstruction obtains the light intensities corresponding to the multiple plane images, and determines the reflectivity and unit normal vector corresponding to the multiple plane images based on the light intensities; obtains the first surface gradient and the second surface gradient corresponding to the multiple plane images according to the reflectivity and the unit normal vector; the first surface gradient and the second surface gradient are the reconstruction elements of the plane image; and obtains the depth map by decomposing the first surface gradient and the second surface gradient and the sparse matrix based on LLT factorization.

[0064] Among them, for object segmentation, the depth map is segmented into regions through a target detection model; the target detection model includes a backbone network, a neck structure and a head structure, the backbone network is used to extract features of the depth map to obtain a feature map, and the neck structure incorporates a coordinate attention mechanism to combine position information and channel attention, which is used to differentially encode the position information of each pixel in the feature map and determine the attention weight of each pixel.

[0065] With respect to the photometric stereo surface defect detection method, system and device provided in the embodiments of the present application, a corresponding partition control image acquisition device is configured to collect multiple sub-images of the chip to be inspected, and the multiple sub-images are stereoscopically reconstructed to obtain a 2.5D image of the chip to be inspected. The 2.5D image is recognized through a target detection model with a coordinate attention mechanism to identify objects with depth values.

[0066] The embodiment of the present application also provides an electronic device, which can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc. The server can be a single server or a server cluster composed of multiple servers, etc.

[0067] In some embodiments, the processing device may also be integrated into multiple electronic devices. For example, the processing device may be integrated into multiple servers, and the surface defect detection method of the present application may be implemented by multiple servers.

[0068] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 5 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0069] The server may include one or more processing core processors 501, one or more computer-readable storage media memories 502, a power supply 503, an input module 504, and a communication module 505. Those skilled in the art will appreciate that Figure 5 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0070] The processor 501 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502. In some embodiments, the processor 501 may include one or more processing cores; in some embodiments, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0071] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0072] The server also includes a power supply 503 for supplying power to various components. In some embodiments, the power supply 503 can be logically connected to the processor 501 through a power management system, so that the power management system can manage charging, discharging, power consumption, etc. The power supply 503 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0073] The server may further include an input module 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0074] The server may also include a communication module 505. In some embodiments, the communication module 505 may include a wireless module. The server may perform short-range wireless transmission through the wireless module of the communication module 505, thereby providing users with wireless broadband Internet access. For example, the communication module 405 may be used to help users send and receive emails, browse web pages, and access streaming media.

[0075] Although not shown, the server may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application programs stored in the memory 502, thereby implementing the steps in the methods of the embodiments of the present application.

[0076] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0077] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0078] For this purpose, see Figure 6 The embodiment of the present application provides a computer-readable storage medium 60, which stores computer-readable instructions 601. The computer-readable instructions 601 include instructions for executing the surface defect detection method described above. For example, the instructions may execute the following steps:

[0079] Perform stereoscopic reconstruction on multiple regional images acquired under light source scenes in different directions to obtain a non-planar image of the object to be detected;

[0080] The non-planar image is segmented into objects, and depth information of the segmented objects is determined, and objects that meet the depth information are determined as target objects.

[0081] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0082] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations of the chip detection aspect provided in the above embodiments.

[0083] Since the instructions stored in the storage medium can execute the steps in any surface defect detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any photometric stereo surface defect detection method, system and device provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0084] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0085] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0086] The above is a detailed introduction to the photometric stereo surface defect detection method, system and equipment provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A photometric stereo surface defect detection method, characterized in that: The method comprises: Stereoscopically reconstruct multiple regional images acquired in light source scenes of different directions to obtain a non-planar image of the object to be detected; the light source scene is a rectangular ring light, and the light source in each direction can be controlled separately; The non-planar image is segmented into objects, and depth information of the segmented objects is determined, and objects that meet the depth information are determined as target objects.

2. The photometric stereo surface defect detection method according to claim 1, characterized in that: The multiple plane images are subjected to stereoscopic reconstruction, comprising: obtaining light intensities corresponding to the multiple plane images, and determining reflectivities and unit normal vectors corresponding to the multiple plane images based on the light intensities; obtaining first surface gradients and second surface gradients corresponding to the multiple plane images according to the reflectivities and the unit normal vectors; the first surface gradients and the second surface gradients are reconstruction elements of the plane images.

3. The photometric stereo surface defect detection method according to claim 2, characterized in that: The non-planar image is a depth map including depth values, and obtaining the non-planar image of the object to be detected includes: obtaining the depth map by factorizing the first surface gradient, the second surface gradient and a sparse matrix based on LLT factorization.

4. The photometric stereo surface defect detection method according to claim 3, characterized in that: The depth map is segmented into regions through a target detection model; the target detection model includes a backbone network, a neck structure and a head structure, the backbone network is used to extract features of the depth map to obtain a feature map, and the neck structure incorporates a coordinate attention mechanism to combine position information and channel attention, which is used to differentially encode the position information of each pixel in the feature map and determine the attention weight of each pixel.

5. The photometric stereo surface defect detection method according to claim 4, characterized in that: The backbone network includes a FasterNet Block structure and a Focus structure; the Focus structure is composed of a convolution block, which is then convolved through a filter, and the final result is a convolution kernel, which has the effect of halving the width and height of the image.

6. The photometric stereo surface defect detection method according to claim 4, characterized in that: The differential encoding process includes: initially encoding the feature map through two one-dimensional encoding processes, then performing global average pooling along the width and height dimensions to split the feature map into different dimensions, and obtaining the corresponding first feature map and second feature map. The first feature map and the second feature map obtain the coordinate attention weight of each channel through the coordinate attention mechanism, and the first feature map and the second feature map are connected to obtain a feature map based on the weight.

7. The photometric stereo surface defect detection method according to claim 6, characterized in that: The head structure is provided with a batch normalization layer and a nonlinear activation layer, which are used to split the connection feature map in width and height dimensions, retain the feature map in the height dimension as the target feature map, and determine the target object based on the depth distribution on the target feature map.

8. A defect detection system, characterized in that: The system includes an image acquisition device and a processing device, the image acquisition device includes an annular light source arranged above the object to be detected and whose center position is coaxial with the center position of the object to be detected, the annular light source includes four sub-light sources, each of which is used to separately acquire a plane image within a corresponding light source angle range; the processing device is used to acquire the plane image and execute the method described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images