Defect detection method and device based on structured light, electronic equipment and storage medium

Abstract

The invention provides a defect detection method and device based on structured light, electronic equipment and a storage medium, and the method comprises the steps: obtaining reference information and a to-be-detected projection image of a to-be-detected circuit board, carrying out the time-frequency signal conversion of the to-be-detected projection image, and obtaining a to-be-detected absolute phase; obtaining a to-be-detected actual phase based on the absolute phase difference between the reference absolute phase and the to-be-detected absolute phase, performing three-dimensional reconstruction according to the to-be-detected actual phase and the equipment calibration parameters to obtain three-dimensional point cloud information, and performing defect detection on the to-be-detected circuit board. The to-be-detected projection image is acquired by image acquisition equipment after a preset structured light pattern is projected to the to-be-detected circuit board through projection equipment, and the equipment calibration parameter is obtained based on calibration of phase coordinate conversion of the projection equipment and the image acquisition equipment; according to the method, the three-dimensional reconstruction speed is improved through the single-frame projection image to be detected, the complexity and the calculation amount of a hardware system are reduced, and a simpler and higher-speed defect detection method is provided.

Description

Technical Field

[0001] The present application relates to the field of precision detection technology, and in particular to a defect detection method, device, electronic device and storage medium based on structured light. Background Art

[0002] Compared with traditional printed circuit board (PCB) defect detection technology, PCB defect detection technology based on three-dimensional (3D) imaging includes the depth information of PCB and can fully describe the location and shape information of components and pads in PCB. Therefore, detection technology based on 3D imaging principle has become the mainstream technology for PCB defect detection. At present, PCB defect detection technology based on 3D imaging principle is mainly divided into three methods: stereo vision, structured light, and a mixture of structured light and stereo vision.

[0003] Among them, stereoscopic vision calculation is complex and difficult, with high requirements for system hardware and high computing power of processing chips. In addition, since the principle of this method is to perform imaging through parallax, the accuracy is low in dim environments and when features are not obvious. The principle of structured light is to project a coded structured light pattern onto the surface of the PCB, collect the surface modulated deformation pattern of the PCB by the image sensor, and further solve the 3D information of the PCB after system calibration to achieve 3D reconstruction. However, at present, this method requires the use of more than 4 structured light patterns for 3D reconstruction, which has a large amount of calculation, low efficiency and slow speed. For example, CN114792305A discloses the generation of point cloud information through FPGA, and the combination of 5 Gray codes and 3 sinusoidal gratings to achieve 3D reconstruction, but the FPGA system is introduced, and a total of 8 structured light patterns need to be projected for reconstruction, which not only increases the hardware cost, but also has low projection efficiency. The purpose of mixing structured light and stereo vision is to improve the low precision of stereo vision in dim environments and when features are not obvious. Although the precision is improved, the amount of calculation is increased, and the optical complexity and hardware cost of the system are also increased. For example, although CN104713885A improves the low precision of stereo vision in dim conditions and when the surface texture of the PCB is sparse, the system consists of at least one projection structured light device and two camera systems, which not only increases the hardware burden and cost, but also increases the amount of calculation. At present, PCB defect detection has the problems of large amount of calculation, low speed, and complex hardware system. Summary of the invention

[0004] The present invention provides a defect detection method, device, electronic device and storage medium based on structured light to solve the technical problems of large calculation amount, low speed and complex hardware system when performing defect detection based on structured light.

[0005] In one embodiment of the present application, the present application provides a defect detection method based on structured light, including: obtaining reference information and a projection image to be inspected of a circuit board to be inspected, the reference information including equipment calibration parameters and a reference absolute phase of a reference plane; performing time-frequency signal transformation on the projection image to be inspected to obtain the absolute phase to be inspected, and obtaining the actual phase to be inspected based on the absolute phase difference between the reference absolute phase and the absolute phase to be inspected; performing three-dimensional reconstruction according to the actual phase to be inspected and the equipment calibration parameters to obtain three-dimensional point cloud information, and performing defect detection on the circuit board to be inspected; wherein, the projection image to be inspected is obtained by projecting a preset structured light pattern onto the circuit board to be inspected through a projection device and then acquiring it through an image acquisition device, and the equipment calibration parameters are obtained based on calibration of the projection device and the image acquisition device for phase coordinate conversion.

[0006] In one embodiment of the present application, a time-frequency signal transformation is performed on the projection image to be tested to obtain an absolute phase to be tested, and based on the absolute phase difference between the reference absolute phase and the absolute phase to be tested, an actual phase to be tested is obtained, including: performing a frequency domain transformation on the projection image to be tested and extracting fundamental frequency information to be tested; performing a time domain transformation on the fundamental frequency information to be tested and extracting the absolute phase to be tested; performing phase unwrapping on the absolute phase difference between the reference absolute phase and the absolute phase to be tested to obtain an actual phase difference; and determining the actual phase to be tested based on the actual phase difference and the reference absolute phase.

[0007] In one embodiment of the present application, the time-frequency signal transformation includes Fourier transform and inverse Fourier transform, or wavelet transform and inverse wavelet transform.

[0008] In one embodiment of the present application, a method for determining a reference absolute phase of a reference plane includes: projecting a preset structured light pattern onto a reference plane through a projection device, and acquiring a reference projection image through an image acquisition device; performing a frequency domain transformation on the reference projection image, and extracting reference fundamental frequency information; performing a time domain transformation on the reference fundamental frequency information, and extracting a reference absolute phase.

[0009] In one embodiment of the present application, defect detection is performed on the circuit board to be inspected, including: projecting depth information onto a two-dimensional plane to generate a depth map, and generating a modulation map based on the modulation information; performing defect identification on the depth map and / or the modulation map to obtain a defect identification result; performing a qualified inspection on the circuit board to be inspected based on the defect identification result to obtain a qualified inspection result; wherein the three-dimensional point cloud information includes the depth information of the circuit board to be inspected and the modulation information.

[0010] In one embodiment of the present application, the present application provides a defect detection device based on structured light, including: an acquisition module, used to acquire reference information and a projection image to be inspected of a circuit board to be inspected, the reference information including equipment calibration parameters and a reference absolute phase of a reference plane; a phase determination module, used to perform time-frequency signal transformation on the projection image to be inspected, obtain the absolute phase to be inspected, and obtain the actual phase to be inspected based on the absolute phase difference between the reference absolute phase and the absolute phase to be inspected; a defect detection module, used to perform three-dimensional reconstruction according to the actual phase to be inspected and the equipment calibration parameters, obtain three-dimensional point cloud information, and perform defect detection on the circuit board to be inspected; wherein, the projection image to be inspected is obtained by projecting a preset structured light pattern onto the circuit board to be inspected through a projection device and then acquiring it through an image acquisition device, and the equipment calibration parameters are obtained based on the calibration of the projection device and the image acquisition device for phase coordinate conversion.

[0011] In one embodiment of the present application, the projection device is also used to project a preset bright light pattern onto the circuit board to be inspected to illuminate the circuit board to be inspected; the image acquisition device is also used to acquire the inspection comparison image of the circuit board to be inspected after illumination, to perform threshold segmentation on the inspection comparison image, and to identify defects based on the result of the threshold segmentation.

[0012] In one embodiment of the present application, the apparatus further includes a conveying device for conveying the circuit board to be inspected to a preset inspection area; the preset inspection area is composed of the image acquisition device, the projection device and the circuit board to be inspected that satisfy a preset triangular relationship.

[0013] In one embodiment of the present application, the present application provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the structured light-based defect detection method as described in any of the above embodiments.

[0014] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the structured light-based defect detection method described in any of the above embodiments.

[0015] Beneficial effects of the embodiments of the present invention: The present invention provides a defect detection method, device, electronic device and storage medium based on structured light. The embodiments of the present invention obtain the actual phase to be inspected by performing time-frequency signal transformation on a single frame of the projection image to be inspected, thereby improving the speed of three-dimensional reconstruction and reducing the complexity and calculation amount of the hardware system, thereby providing a simpler and faster defect detection method.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present application can be applied;

[0019] Figure 2 A schematic diagram of a defect detection method based on structured light according to an embodiment of the present application is shown;

[0020] FIG3( a ) shows a schematic diagram of vertical grating stripes in a structured light pattern according to an embodiment of the present application;

[0021] FIG3( b ) shows a schematic diagram of horizontal grating stripes in a structured light pattern according to an embodiment of the present application;

[0022] FIG3( c ) shows a schematic diagram of circular grating stripes in a structured light pattern according to an embodiment of the present application;

[0023] Figure 4 A schematic diagram of a defect detection device based on structured light according to an embodiment of the present application is shown;

[0024] Figure 5 A signal control schematic diagram according to an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram showing a PCB defect according to an embodiment of the present application is shown;

[0026] Figure 7 A schematic diagram of an implementation process of a defect detection method based on structured light according to an embodiment of the present application is shown;

[0027] Figure 8 A block diagram of a defect detection device based on structured light according to an embodiment of the present application is shown;

[0028] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0030] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present application in a schematic manner. Therefore, the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0031] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0032] See also Figure 1 , Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied is shown. Figure 1As shown, the system architecture may include a computer device 101, an image acquisition device 102, a projection device 103, a circuit board to be inspected 104 and a stage 105. The computer device 101 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, a general-purpose computer, etc.; the image acquisition device 102 includes an image sensor or a signal acquisition circuit with an image acquisition function; the projection device 103 includes a laser scanning projection device such as a liquid crystal on silicon (LCoS), a digital light processing (DLP), a liquid crystal display (LCD), a micro-electro-mechanical system (MEMS), etc. with a digital grating projection function; the circuit board to be inspected 104 is located on the stage 105; the image acquisition device 102, the projection device 103 and the circuit board to be inspected 104 are in a preset triangle relationship. The projection device 103 is used to project a preset structured light pattern onto the circuit board 104 to be inspected, and the image acquisition device 102 is used to acquire the structured light pattern modulated by the circuit board 104 to be inspected, obtain the projected image to be inspected, and transmit it to the computer device 101 for defect detection.

[0033] Exemplarily, the computer device 101 obtains reference information and a projection image to be inspected of the circuit board to be inspected, the reference information including device calibration parameters and a reference absolute phase of a reference plane; performs time-frequency signal transformation on the projection image to be inspected to obtain the absolute phase to be inspected, and obtains the actual phase to be inspected based on the absolute phase difference between the reference absolute phase and the absolute phase to be inspected; performs three-dimensional reconstruction according to the actual phase to be inspected and the device calibration parameters to obtain three-dimensional point cloud information, and performs defect detection on the circuit board to be inspected; wherein, the projection image to be inspected is obtained by projecting a preset structured light pattern onto the circuit board to be inspected through a projection device and then acquiring it through an image acquisition device, and the device calibration parameters are obtained based on the calibration of the projection device and the image acquisition device for phase coordinate conversion.

[0034] In the related technologies, when defect detection is performed based on structured light, there are technical problems such as large amount of calculation, low speed and complex hardware system.

[0035] In order to solve the above technical problems, the present application provides a defect detection method, device, electronic device and storage medium based on structured light. The implementation details of the technical solution of the embodiment of the present application are elaborated in detail below.

[0036] See also Figure 2 , Figure 2FIG. 2 shows a flow chart of a defect detection method based on structured light according to an embodiment of the present application. Figure 2 As shown, in an exemplary embodiment, the defect detection method based on structured light at least includes steps S210 to S230, which are described in detail as follows:

[0037] Step S210, obtaining reference information and a projection image of the circuit board to be inspected.

[0038] Among them, the reference information includes device calibration parameters and a reference absolute phase of a reference plane. The projection image to be tested is obtained by projecting a preset structured light pattern onto the circuit board to be tested through a projection device and then acquiring it through an image acquisition device. The device calibration parameters are obtained based on the calibration of phase coordinate conversion of the projection device and the image acquisition device.

[0039] In one embodiment of the present application, the circuit boards to be inspected include various types of circuit boards in the electronics and mixed assembly industries, such as printed circuit boards (PCBs) and aluminum substrates.

[0040] In one embodiment of the present application, a computer device controls a projection device to project a preset structured light pattern onto the circuit board to be inspected, wherein the preset structured light pattern is a digital grating pattern, including a periodic phase rule, and may be circular, horizontal, vertical stripes, etc.

[0041] In one embodiment of the present application, please refer to Figures 3(a) to 3(c), Figure 3(a) shows a schematic diagram of vertical grating stripes in a structured light pattern according to one embodiment of the present application, Figure 3(b) shows a schematic diagram of horizontal grating stripes in a structured light pattern according to one embodiment of the present application, and Figure 3(c) shows a schematic diagram of circular grating stripes in a structured light pattern according to one embodiment of the present application. As shown in Figures 3(a) to 3(c), the preset structured light pattern can be a grating stripe pattern with periodic information, such as the vertical grating stripes in Figure 3(a), the horizontal grating stripes in Figure 3(b), and the circular grating stripes in Figure 3(c).

[0042] In one embodiment of the present application, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a defect detection device based on structured light according to an embodiment of the present application. Figure 4As shown, the defect detection device based on structured light of the present application includes a computer device 401, an image acquisition device 402, a projection device 403, a circuit board to be inspected 404, a stage 405 and a conveying device 406; wherein, the circuit board to be inspected 404 is conveyed to a preset detection area by the conveying device 406; the preset detection area is composed of an image acquisition device 402, a projection device 403 and a circuit board to be inspected 404 that satisfy a preset triangular relationship; the computer device 401 is hardware-connected with the image acquisition device 402, the projection device 403 and the conveying device 406 respectively, and provides a synchronous modulation signal; the computer device 401 also includes a digital grating writing of the projection device; according to the principle of the synchronous modulation signal, the computer device 401 sends a trigger signal to the conveying device 406, so as to trigger the projection device 403 to project a preset structured light pattern onto the circuit board to be inspected 404 when the circuit board to be inspected is sent to the preset detection area, and trigger the image acquisition device 402 to capture an image and store it in the computer device 401 for defect detection.

[0043] In one embodiment of the present application, please continue to refer to Figure 4 , and see Figure 5 , Figure 5 FIG. 2 shows a schematic diagram of signal control according to an embodiment of the present application. Figure 4 and Figure 5 As shown, the conveying device 406 includes a movable platform or carrier, and the synchronous modulation signal is a pulse signal; the computer device 401 generates a first signal to control the movement of the carrier of the circuit board 404 to be inspected. When the circuit board 404 to be inspected moves to a preset detection area, the computer device 401 generates a second signal to control the projection device 403 to project a preset structured light pattern onto the circuit board 404 to be inspected. After a delay time t, a third signal is generated to control the image acquisition device 402 to acquire the modulated structured light pattern of the circuit board 404 to be inspected, and store it in the computer device 401.

[0044] In one embodiment of the present application, determination of equipment calibration parameters includes phase height method and inverse trigonometric method.

[0045] In one embodiment of the present application, determining the device calibration parameters includes establishing a phase-height conversion relationship through a phase-height method based on the internal and external parameters of the projection device and the image acquisition device to obtain a three-dimensional coordinate conversion relationship; or, establishing a phase-three-dimensional coordinate conversion relationship through an inverse trigonometry method based on the internal and external parameters of the projection device and the image acquisition device.

[0046] In one embodiment of the present application, the internal parameters include focal length, principal point offset, radial distortion coefficient, and tangential distortion coefficient to describe the internal geometry and optical properties of the corresponding device; the external parameters include rotation matrix and translation vector to describe the relationship between the corresponding device and the world coordinate system.

[0047] Step S220 , performing time-frequency signal transformation on the projection image to be tested to obtain an absolute phase to be tested, and obtaining an actual phase to be tested based on an absolute phase difference between the reference absolute phase and the absolute phase to be tested.

[0048] In one embodiment of the present application, a time-frequency signal transformation is performed on a projection image to be tested to obtain an absolute phase to be tested, and based on the absolute phase difference between a reference absolute phase and the absolute phase to be tested, an actual phase to be tested is obtained, including: performing a frequency domain transformation on the projection image to be tested and extracting fundamental frequency information to be tested; performing a time domain transformation on the fundamental frequency information to be tested and extracting the absolute phase to be tested; performing phase unwrapping on the absolute phase difference between the reference absolute phase and the absolute phase to be tested to obtain an actual phase difference; and determining the actual phase to be tested according to the actual phase difference and the reference absolute phase.

[0049] In one embodiment of the present application, the time-frequency signal transformation includes Fourier transform and inverse Fourier transform, or wavelet transform and inverse wavelet transform.

[0050] In one embodiment of the present application, a method for determining a reference absolute phase of a reference plane includes: projecting a preset structured light pattern onto a reference plane through a projection device, and acquiring a reference projection image through an image acquisition device; performing a frequency domain transformation on the reference projection image, and extracting reference fundamental frequency information; performing a time domain transformation on the reference fundamental frequency information, and extracting a reference absolute phase.

[0051] In one embodiment of the present application, the reference plane includes a plane for reference, such as a white paper plane, wherein the reference plane can be placed in a preset detection area to project a preset structured light pattern.

[0052] In one embodiment of the present application, Fourier transform is performed on the reference projection image and the projection image to be tested, and then the fundamental frequency of the spectrum is extracted using a window algorithm. The reference fundamental frequency information and the fundamental frequency information to be tested are inverse Fourier transformed and then subtracted to obtain an absolute phase difference. The absolute phase difference is phase unwrapped to obtain an actual phase difference, thereby obtaining the actual phase to be tested of the projection image to be tested.

[0053] In one embodiment of the present application, the window algorithm includes a rectangular window, a Gaussian window, a Hanning window, and the like.

[0054] In one embodiment of the present application, the phase unwrapping algorithm includes a mass method and a tree branch method.

[0055] Step S230, performing three-dimensional reconstruction according to the actual phase to be inspected and the equipment calibration parameters to obtain three-dimensional point cloud information, and performing defect detection on the circuit board to be inspected.

[0056] In one embodiment of the present application, defect detection is performed on a circuit board to be inspected, including: projecting depth information onto a two-dimensional plane to generate a depth map, and generating a modulation map based on the modulation information; performing defect identification on the depth map and / or the modulation map to obtain a defect identification result; performing a qualified inspection on the circuit board to be inspected based on the defect identification result to obtain a qualified inspection result; wherein the three-dimensional point cloud information includes the depth information and modulation information of the circuit board to be inspected.

[0057] In one embodiment of the present application, the three-dimensional point cloud information not only includes the depth information of the circuit board to be inspected, but also includes the modulation information of the circuit board to be inspected, which can further improve the speed of defect detection. The defect recognition results identified by the depth information include the defect position, range, size and warping of the circuit board to be inspected, component size and other information of the circuit board to be inspected. Therefore, after obtaining the depth information, not only can it be detected whether there are defects, but also by measuring the defect range, size and other information, it can be determined whether the defects of the PCB board are acceptable and whether the quality is qualified. The modulation information is a parameter for brightness contrast in different areas. The defect position can be quickly identified by comparing the modulation.

[0058] In one embodiment of the present application, at least one of independent defect detection can be performed based on depth information, independent defect detection can be performed based on modulation information, and joint defect detection can be performed based on depth information and modulation information.

[0059] In one embodiment of the present application, see Figure 6 , Figure 6 FIG. 1 is a schematic diagram showing a PCB defect according to an embodiment of the present application. Figure 6 As shown, a preset structured light pattern 621 is projected onto a printed circuit board 630 by a projection device 620. After being collected by an image acquisition device 610, three-dimensional point cloud information is obtained. The three-dimensional point cloud information can be used to locate defects, measure defect sizes, and scratch depths, such as device deviation from pads 631, device package damage 632, excess residue 633, and pad scratches 634, to determine whether the quality of the PCB being tested is qualified, thereby improving product quality. The preset structured light pattern 621 includes vertical grating stripes.

[0060] In one embodiment of the present application, please continue to refer to Figure 1 ,like Figure 1As shown, the projection device 103 can be used to project a pure white structured light pattern to provide lighting conditions for the circuit board 104 to be inspected, so as to improve the contrast when the image acquisition device 402 captures the image. Compared with the two-dimensional defect detection method without lighting, the accuracy is improved and the amount of calculation is reduced, which makes it easier to implement threshold segmentation of the comparison image to be inspected, and locate the defect position more quickly, such as component offset and damage; threshold segmentation includes binarization, grayscale, edge processing and other algorithms. When using the two-dimensional defect detection method, the defect detection device provided by the present application can also act as a multimedia camera device, and its captured image is clearer than that of ordinary multimedia camera equipment, so as to improve the traceability of product manufacturing. In addition, the present application can integrate three-dimensional defect detection and two-dimensional defect detection, providing a multi-degree-of-freedom solution for defect detection.

[0061] In one embodiment of the present application, see Figure 7 , Figure 7 FIG. 1 is a schematic diagram showing an implementation process of a defect detection method based on structured light according to an embodiment of the present application. Figure 7 As shown, by performing Fourier transform on the reference surface structured light pattern and the PCB board structured light pattern acquired by the image acquisition device respectively, the fundamental frequency is extracted, and the phase is extracted after Fourier inverse transform according to the fundamental frequency information. The absolute phase of the reference surface and the absolute phase of the PCB board are subtracted to obtain the absolute phase difference. The absolute phase difference is phase-unwrapped to obtain the actual phase of the PCB board. The PCB board three-dimensional point cloud is generated in combination with the system calibration parameters, and the surface of the PCB board is defect-detected. A single frame of structured light pattern can realize the three-dimensional shape reconstruction of the PCB board surface, providing a simpler and faster solution for the PCB defect detection method based on structured light.

[0062] See also Figure 8 , Figure 8 A block diagram of a defect detection device based on structured light according to an embodiment of the present application is shown. The device can be applied to Figure 1 The implementation environment shown in the figure is specifically configured in the computer device 101. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.

[0063] like Figure 8 As shown, a defect detection device 800 based on structured light according to an embodiment of the present application includes: an acquisition module 801 , a phase determination module 802 and a defect detection module 803 .

[0064] An acquisition module 801 is used to acquire reference information and a projection image of the circuit board to be inspected, wherein the reference information includes a device calibration parameter and a reference absolute phase of a reference surface;

[0065] The phase determination module 802 is used to perform time-frequency signal transformation on the projection image to be tested to obtain the absolute phase to be tested, and obtain the actual phase to be tested based on the absolute phase difference between the reference absolute phase and the absolute phase to be tested;

[0066] The defect detection module 803 is used to perform three-dimensional reconstruction according to the actual phase to be inspected and the equipment calibration parameters, obtain three-dimensional point cloud information, and perform defect detection on the circuit board to be inspected;

[0067] Among them, the projection image to be tested is obtained by projecting a preset structured light pattern onto the circuit board to be tested through a projection device and then acquiring it through an image acquisition device, and the device calibration parameters are obtained by calibrating the phase coordinate conversion of the projection device and the image acquisition device.

[0068] In one embodiment of the present application, the projection device is also used to project a preset bright light pattern onto the circuit board to be inspected so as to illuminate the circuit board to be inspected; the image acquisition device is also used to acquire the inspection comparison image of the circuit board to be inspected after illumination so as to perform threshold segmentation on the inspection comparison image and perform defect identification based on the result of the threshold segmentation.

[0069] In one embodiment of the present application, the structured light-based defect detection device also includes a conveying device for conveying the circuit board to be inspected to a preset inspection area; the preset inspection area is composed of an image acquisition device, a projection device and the circuit board to be inspected that satisfy a preset triangular relationship.

[0070] It should be noted that the defect detection device based on structured light provided in the above embodiment and the defect detection method based on structured light provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the defect detection device based on structured light provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0071] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device implements the structured light-based defect detection method provided in the above-mentioned embodiments.

[0072] See also Fig. 9 , Fig. 9 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig. 9The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0073] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 to the random access memory (RAM) 903, such as executing the method in the above embodiment. In the RAM 903, various programs and data required for system operation are also stored. The CPU 901, ROM 902 and RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0074] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0075] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 909, and / or installed from a removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, various functions defined in the system of the present application are executed.

[0076] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0077] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The units involved in the embodiments described in the present application can be implemented by software or by hardware, and the units described can also be set in a processor. Among them, the names of these units do not constitute a limitation on the units themselves under certain circumstances. Therefore, the technical solution according to the implementation mode of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation mode of the present application.

[0079] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the defect detection method based on structured light provided in the above-mentioned embodiments. The computer-readable storage medium may be included in the electronic device described in the above-mentioned embodiments, or may exist independently without being assembled into the electronic device.

[0080] In the above embodiments, unless otherwise specified, by using serial numbers such as "first" and "second" to describe common objects, it only means that they refer to different instances of the same object, rather than indicating that the objects being described must adopt a given order, whether in time, space, sorting or any other way.

[0081] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A defect detection method based on structured light, characterized in that: The method comprises: Acquire reference information and a projection image of the circuit board to be inspected, wherein the reference information includes equipment calibration parameters and a reference absolute phase of a reference surface; Performing a time-frequency signal transformation on the projection image to be tested to obtain an absolute phase to be tested, and obtaining an actual phase to be tested based on an absolute phase difference between the reference absolute phase and the absolute phase to be tested; Performing three-dimensional reconstruction according to the actual phase to be inspected and the calibration parameters of the device to obtain three-dimensional point cloud information, and performing defect detection on the circuit board to be inspected; Among them, the projection image to be tested is obtained by projecting a preset structured light pattern onto the circuit board to be tested through a projection device and then acquiring it through an image acquisition device, and the device calibration parameters are obtained based on the calibration of phase coordinate conversion of the projection device and the image acquisition device.

2. The defect detection method based on structured light according to claim 1, characterized in that: Performing a time-frequency signal transformation on the projection image to be tested to obtain an absolute phase to be tested, and obtaining an actual phase to be tested based on an absolute phase difference between the reference absolute phase and the absolute phase to be tested, including: Performing frequency domain transformation on the projection image to be tested, and extracting fundamental frequency information to be tested; Performing time domain transformation on the baseband information to be detected, and extracting the absolute phase to be detected; Performing phase unwrapping on the absolute phase difference between the reference absolute phase and the absolute phase to be detected to obtain an actual phase difference; The actual phase to be detected is determined according to the actual phase difference and the reference absolute phase.

3. The defect detection method based on structured light according to claim 2, characterized in that: The time-frequency signal transformation includes Fourier transform and inverse Fourier transform, or wavelet transform and inverse wavelet transform.

4. The defect detection method based on structured light according to any one of claims 1 to 3, characterized in that: The reference absolute phase of the reference surface is determined by: Projecting a preset structured light pattern onto a reference surface through a projection device, and acquiring a reference projection image through an image acquisition device; Performing frequency domain transformation on the reference projection image and extracting reference fundamental frequency information; The reference baseband information is transformed in the time domain, and a reference absolute phase is extracted.

5. The defect detection method based on structured light according to any one of claims 1 to 3, characterized in that: Performing defect detection on the circuit board to be inspected includes: Projecting the depth information onto a two-dimensional plane to generate a depth map, and generating a modulation map based on the modulation information; Performing defect recognition on the depth map and / or modulation map to obtain a defect recognition result; Based on the defect identification result, a qualified test is performed on the circuit board to be tested to obtain a qualified test result; The three-dimensional point cloud information includes the depth information of the circuit board to be inspected and the modulation information.

6. A defect detection device based on structured light, characterized in that: The device comprises: An acquisition module, used to acquire reference information and a projection image of the circuit board to be inspected, wherein the reference information includes a device calibration parameter and a reference absolute phase of a reference surface; A phase determination module, used for performing time-frequency signal transformation on the projection image to be tested to obtain an absolute phase to be tested, and obtaining an actual phase to be tested based on an absolute phase difference between the reference absolute phase and the absolute phase to be tested; A defect detection module, used to perform three-dimensional reconstruction according to the actual phase to be inspected and the calibration parameters of the device, obtain three-dimensional point cloud information, and perform defect detection on the circuit board to be inspected; Among them, the projection image to be tested is obtained by projecting a preset structured light pattern onto the circuit board to be tested through a projection device and then acquiring it through an image acquisition device, and the device calibration parameters are obtained based on the calibration of phase coordinate conversion of the projection device and the image acquisition device.

7. The defect detection device based on structured light according to claim 6, characterized in that: The projection device is also used to project a preset bright light pattern onto the circuit board to be inspected, so as to illuminate the circuit board to be inspected; The image acquisition device is also used to acquire the comparison image to be inspected of the illuminated circuit board to be inspected, so as to perform threshold segmentation on the comparison image to be inspected, and to perform defect recognition according to the result of the threshold segmentation.

8. The defect detection device based on structured light according to any one of claims 6 to 7, characterized in that: The device also includes a conveying device for conveying the circuit board to be inspected to a preset inspection area; The preset detection area is composed of the image acquisition device, the projection device and the circuit board to be inspected which satisfy a preset triangular relationship.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the structured light-based defect detection method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the defect detection method based on structured light according to any one of claims 1 to 5.

Images