PCB image correction method and device, terminal equipment and storage medium

Abstract

Embodiments of the present application disclose a PCB image correction method and device, a terminal device and a storage medium. In the method, a corresponding substrate area is confirmed according to a depth map and a two-dimensional map of a to-be-tested PCB; three-dimensional coordinate data of the substrate area is confirmed based on depth information of the depth map and coordinates of the two-dimensional map; a reference surface is obtained by fitting a set number of points randomly selected from the three-dimensional coordinate data; and the depth of each point in the depth map is corrected based on the reference surface and a preset reference plane as a reference to obtain a corrected depth map. By comprehensively using depth information in the depth map and color information in the two-dimensional map, the substrate area is confirmed, the three-dimensional coordinate data of the substrate area is fitted, and the depth of each point in the depth map is corrected based on the preset reference plane as a reference, so that the image obtained in the optical detection process is guaranteed, and the most accurate depth data can be finally obtained, thereby effectively reducing the occurrence of false detection results.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of optical quality inspection technology, and in particular to PCB image correction methods, apparatus, terminal equipment and storage media. Background Technology

[0002] In the field of 3D Automated Optical Inspection (3DAOI), target objects such as PCBs (Printed Circuit Boards) are scanned using 3D imaging methods such as structured light to obtain depth information. This information is then combined with 2D texture images captured by a camera and compared with standard qualified data in a database to detect defects in the PCB. Typically, 3DAOI machines are pre-set with a reference plane (0 height), which can be considered parallel to the machine's guide plane. The depth information ultimately acquired by the sensor module is the relative depth with respect to this reference plane.

[0003] When using existing testing equipment for optical inspection, the inventors found that due to factors such as the bending of PCB component pins and substrate, the PCB placed on the guide rail may not be parallel to the reference plane, which can lead to a large difference between the depth obtained and the required depth, resulting in false detections. Summary of the Invention

[0004] This invention provides a PCB image correction method, apparatus, device, and storage medium to solve the technical problem of inaccurate PCB inspection due to deformation or tilt in the prior art.

[0005] In a first aspect, embodiments of the present invention provide a PCB image correction method, comprising:

[0006] Identify the corresponding substrate area based on the depth map and 2D map of the PCB under test;

[0007] Based on the depth information of the depth map and the coordinates of the two-dimensional map, the three-dimensional coordinate data of the substrate region is confirmed;

[0008] A reference surface is obtained by randomly selecting a set number of points from the three-dimensional coordinate data and fitting them.

[0009] Based on the reference plane, and with a preset reference plane as a reference, the depth of each point in the depth map is corrected to obtain the corrected depth map.

[0010] Secondly, embodiments of the present invention provide a PCB image correction method, including:

[0011] Identify the corresponding substrate area based on the depth map and 2D map of the PCB under test;

[0012] The depth information associated with the depth map and the substrate region, and the pixel coordinates associated with the substrate region in the two-dimensional map are used as samples to train a BP neural network model. The BP neural network model is used to output the height value of the pixel coordinates corresponding to a preset reference surface.

[0013] The pixel coordinates in the two-dimensional image are input into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;

[0014] The difference between the depth information and the corresponding height value of the depth map is used as the corrected depth information.

[0015] Thirdly, embodiments of the present invention also provide a PCB image correction device, comprising:

[0016] The substrate area determination unit is used to determine the corresponding substrate area based on the depth map and two-dimensional map of the PCB under test.

[0017] The coordinate data confirmation unit is used to confirm the three-dimensional coordinate data of the substrate area based on the depth information of the depth map and the coordinates of the two-dimensional map.

[0018] The coordinate data fitting unit is used to randomly select a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface;

[0019] The depth map correction unit is used to correct the depth of each point in the depth map based on the reference plane and with a preset reference plane as a reference, so as to obtain a corrected depth map.

[0020] Fourthly, embodiments of the present invention also provide a PCB image correction device, comprising:

[0021] The substrate area determination unit is used to determine the corresponding substrate area based on the depth map and two-dimensional map of the PCB under test.

[0022] The model training unit is used to train a BP neural network model by using the depth information associated with the depth map and the substrate region, as well as the pixel coordinates associated with the substrate region in the two-dimensional map, as samples. The BP neural network model is used to output the height value of the pixel coordinates corresponding to a preset reference surface.

[0023] The height value output unit is used to input the pixel coordinates in the two-dimensional image into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;

[0024] The depth information correction unit is used to take the difference between the depth information and the corresponding height value of the depth map as the corrected depth information.

[0025] Fifthly, embodiments of the present invention also provide a terminal device, including:

[0026] One or more processors;

[0027] Memory, used to store one or more programs;

[0028] When the one or more programs are executed by the one or more processors, the terminal device implements the PCB image correction method as described in the first or second aspect.

[0029] In a sixth aspect, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the PCB image correction method as described in the first or second aspect.

[0030] The aforementioned PCB image correction method, apparatus, terminal device, and storage medium, in this method, identify the corresponding substrate region based on the depth map and two-dimensional map of the PCB under test; based on the depth information of the depth map and the coordinates of the two-dimensional map, identify the three-dimensional coordinate data of the substrate region; randomly select a predetermined number of points from the three-dimensional coordinate data for fitting to obtain a reference plane; based on the reference plane, and with a preset reference plane as a reference, correct the depth of each point in the depth map to obtain a corrected depth map. By integrating the depth information in the depth map and the color information in the two-dimensional map to identify the substrate region, fitting the reference plane based on the three-dimensional coordinate data of the substrate region, and correcting the depth of each point in the depth map based on the reference plane and with a preset reference plane as a reference, this method ensures that the image obtained during optical inspection can ultimately obtain the most accurate depth data possible, effectively reducing the occurrence of false detection results. Attached Figure Description

[0031] Figure 1 This is a flowchart of a PCB image correction method provided in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the substrate tilted.

[0033] Figure 3 This is a schematic diagram illustrating the correction of a PCB image with a plane as the reference plane, provided in an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram illustrating the correction of a curved reference surface in a PCB image provided in an embodiment of the present invention.

[0035] Figure 5 This is a flowchart of a PCB image correction method provided in an embodiment of the present invention;

[0036] Figure 6 Here is a schematic diagram of the structure of the BP neural network model provided in this embodiment of the invention;

[0037] Figure 7 This is a schematic diagram of the structure of a PCB image correction device provided in an embodiment of the present invention;

[0038] Figure 8 This is a schematic diagram of the structure of a PCB image correction device provided in an embodiment of the present invention;

[0039] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0040] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and not for limiting the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention and not the entire structure.

[0041] It should be noted that, due to space limitations, this application specification does not exhaustively list all possible implementation methods. Those skilled in the art should be able to conceive after reading this application specification that, as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0042] For example, in one embodiment, a technical feature is described: the first substrate region is obtained by segmenting the depth map using the depth information of the depth map. In another embodiment, another technical feature is described: the second substrate region in the two-dimensional image is confirmed by statistical analysis of the color channels in the two-dimensional image. Those skilled in the art should be able to conceive of this application specification after reading it. An implementation that has both of these features is also an optional implementation. That is, in the specific implementation process, the first substrate region is obtained based on the depth information, and the second substrate region is obtained based on the statistical analysis of the color channels, thereby confirming the substrate region of the PCB to be tested.

[0043] The embodiments are described in detail below.

[0044] Figure 1 This is a flowchart of a PCB image correction method provided in an embodiment of the present invention. The PCB image correction method is used in a terminal device. As shown in the figure, the PCB image correction method includes:

[0045] Step S110: Confirm the corresponding substrate area based on the depth map and 2D map of the PCB to be tested.

[0046] In actual production, to reduce production costs, a large number of PCBs of a particular type are typically produced at once. This means a batch of PCBs to be tested have almost identical surface shapes and layouts. Based on this, when the first product is produced, it can be used as a sample of the PCB to be tested, and the substrate area can be marked on its depth map and 2D image. Subsequent image corrections are then performed using this substrate area. After each PCB is completed and enters the testing equipment, the equipment scans it using 3D imaging methods such as structured light, generating a depth map and a 2D image. The substrate area is then marked on these depth maps and / or 2D images.

[0047] In the specific implementation process, step S110 can also be implemented through steps S111-S113:

[0048] Step S111: Preprocess the depth map of the PCB to be tested to obtain the first substrate area.

[0049] When preprocessing the depth map of the PCB under test, the main process involves segmenting the depth map based on a preset height threshold and extracting the portion of the depth map below the height threshold to obtain the first substrate region. In other words, by processing the depth map, regions corresponding to higher components on the PCB, such as chip regions and capacitor regions, are removed.

[0050] Step S112: Preprocess the two-dimensional image of the PCB to be tested to obtain the second substrate area.

[0051] The preprocessing of the 2D image of the PCB to be tested mainly involves identifying the area corresponding to the substrate based on the color characteristics of the PCB. For the PCB, the substrate area is generally the mask area corresponding to green. At this time, the continuously distributed green area can be identified as the mask area.

[0052] In the specific implementation process, step S112 can also be implemented through steps S1121-S1123:

[0053] Step S1121: Perform grayscale histogram statistics on the three color channels of the two-dimensional image.

[0054] Step S1122: Based on the preset grayscale range value, confirm the undetermined mask area corresponding to each color channel.

[0055] Step S1123: Obtain the second substrate region based on the intersection of the undetermined mask regions corresponding to each color channel.

[0056] The acquired two-dimensional image is actually the RGB texture image of the PCB under test. Gray-level histogram statistics are performed on the three color channels R, G and B respectively. Based on the set gray-level range values, the gray-level range with the most pixels [I1,I2] is obtained, and then the corresponding undetermined mask area mask(i,j) is obtained.

[0057]

[0058] The R, G, and B color channels can be used to calculate the corresponding mask area to be determined. R (i,j), mask G (i,j) and mask B (i,j), the final mask region (i.e., the second substrate region) is:

[0059] mask = mask R (i,j)&mask G (i,j)&mask B (i,j).

[0060] Step S113: Based on the intersection of the first substrate region and the second substrate region, confirm the substrate region.

[0061] In the specific processing of the intersection region, there can be different variations in the processing details. For example, the calculation of the first substrate region and the second substrate region can be performed separately, and then the intersection of the first substrate region and the second substrate region can be taken; or the calculation of the first substrate region can be performed first, and the calculation of the second substrate region can be performed within the range of the first substrate region. That is, based on the first substrate region obtained by segmentation based on the height threshold, only the two-dimensional image corresponding to this region is preprocessed to obtain the second substrate region. In this case, the intersection of the three channels is the second substrate region, which also corresponds to the final substrate region.

[0062] Step S120: Based on the depth information of the depth map and the coordinates of the two-dimensional map, confirm the three-dimensional coordinate data of the substrate area.

[0063] From the depth information obtained in the original depth map, the z-coordinates of three-dimensional points in the substrate region can be directly obtained. Additionally, in the inspection equipment, two-dimensional... Figure 1 Generally, the data is acquired through a telecentric lens; the pixel coordinates of the two-dimensional image are proportionally converted based on the parameters of the telecentric lens to obtain the x and y coordinates in the three-dimensional coordinate data, and the corresponding x, y, and z coordinates are used as the three-dimensional coordinate data of the three-dimensional points in the substrate area.

[0064] Step S130: Randomly select a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface.

[0065] In this scheme, for a large amount of three-dimensional coordinate data corresponding to the substrate area, instead of fitting each point to a surface, a set number of points are randomly selected for fitting to obtain a reference surface.

[0066] To ensure accurate fitting results, multiple sets of 3D coordinate data can be randomly selected for multiple fittings, and the optimal result can be chosen as the reference surface. In other words, in the specific implementation process, step S130 can be carried out through steps S131-S134:

[0067] Step S131: Randomly select a preset number of points from the three-dimensional coordinate data for fitting to obtain at least two initial reference planes.

[0068] Step S132: Calculate the distance from each point to each initial reference plane based on the three-dimensional coordinate data.

[0069] Step S133: Count the valid points corresponding to each initial reference plane. The valid points are the points whose distance from the initial reference plane is less than a preset distance threshold.

[0070] Step S134: Use the initial reference plane with the most effective points as the reference plane of the substrate region.

[0071] The fitting of each set of 3D coordinate data follows the same method as existing fitting techniques. For the multiple initial reference planes obtained through fitting, the distance between the 3D coordinate data and each initial reference plane is further calculated. Points with a distance less than a preset distance threshold are recorded as valid points on that initial reference plane. The more valid points on an initial reference plane, the closer the fitting result is to the distribution of points. Therefore, the initial reference plane with the most valid points is used as the reference plane for the substrate region. Through multiple random samplings, the influence of outliers on the final fitting result can be effectively avoided, resulting in a more accurate result. The initial reference planes obtained through fitting and the finally confirmed reference planes are described by spatial equations.

[0072] Step S140: Based on the reference plane and with a preset reference plane as a reference, correct the depth of each point in the depth map to obtain the corrected depth map.

[0073] The reference surface is either a plane or a curved surface. The spatial equation of the plane is k0x + k1y + z + k2 = 0, and the spatial equation of the curved surface is k3x. 2 +k4y 2 +k5xy+k6x+k7y+k8+z=0;

[0074] The step of correcting the depth of each point in the depth map based on the reference plane and using a preset reference plane as a reference to obtain a corrected depth map includes:

[0075] If the reference plane is a plane, the points in the depth map are corrected using the following first depth correction formula:

[0076]

[0077] If the reference surface is a curved surface, the points in the depth map are corrected using the following second depth correction formula:

[0078]

[0079] Where (i,j) are the pixel coordinates corresponding to the midpoint (x,y) in the three-dimensional coordinate data, h'(i,j) are the correction depth corresponding to the midpoint (x,y) in the three-dimensional coordinate data, h(i,j) are the measurement depth corresponding to the midpoint (x,y) in the three-dimensional coordinate data, mi = x, mj = y, and m is the object resolution (um / pixel) of the camera that acquires the two-dimensional image.

[0080] For the plane k0x+k1y+z+k2=0, we can establish as follows: Figure 3 The calibration model shown has a fitted reference plane Π1, a reference plane Π0, and a normal vector of the reference plane. The normal vector of the reference plane is Let the angle between the two planes be θ, then we can calculate:

[0081]

[0082] like Figure 3 As shown, the actual height of a point on the PCB relative to the reference plane is h', but the height measured by the testing equipment is h. From the spatial equation of the reference plane, we can obtain:

[0083]

[0084] In the three-dimensional coordinate data, (x, y) are physical coordinates. Representing the physical coordinates (x, y) using pixel coordinates (i, j), where mi = x, mj = y, (i, j) are pixel coordinates, and m is the object-space resolution of the camera acquiring the two-dimensional image, we can obtain the first depth correction formula when the reference plane is a plane:

[0085]

[0086] For the surface k3x 2 +k4y 2 +k5xy+k6x+k7y+k8+z=0, can establish as follows Figure 4 The calibration model shown has a fitted reference plane S1, a reference plane Π0, and a normal vector of the reference plane. The angle between the actual height h' and the measured height h is θ. Since θ is generally smaller than θ during PCB substrate calibration, the normal vector at h' can be approximated as the normal vector of the surface at the measurement position, i.e. Further calculations can be performed to obtain:

[0087]

[0088] and Figure 3 The corresponding calculation method is similar:

[0089]

[0090] Finally, the second correction formula is obtained:

[0091]

[0092] The above describes the process of identifying the corresponding substrate region based on the depth map and 2D map of the PCB under test; determining the 3D coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the 2D map; randomly selecting a set number of points from the 3D coordinate data for fitting to obtain a reference plane; and correcting the depth of each point in the depth map based on the reference plane and a preset reference plane to obtain a corrected depth map. By combining the depth information in the depth map and the color information in the 2D map, the substrate region is identified. A reference plane is fitted based on the 3D coordinate data of the substrate region. Using the reference plane and a preset reference plane as a reference, while correcting the reference plane to the reference plane, other regions are also corrected to their corresponding height values. This process corrects the depth of each point in the depth map, ensuring that the image obtained during optical inspection yields the most accurate depth data possible and effectively reducing false detection results.

[0093] Figure 5 A flowchart of another PCB image correction method provided in an embodiment of the present invention is shown in the figure. The PCB image correction method includes:

[0094] Step S210: Confirm the corresponding substrate area based on the depth map and 2D map of the PCB to be tested.

[0095] Step S220: The depth information associated with the depth map and the substrate region, and the pixel coordinates associated with the substrate region in the two-dimensional map are used as samples to train a BP neural network model. The BP neural network model is used to output the height value of the pixel coordinates corresponding to a preset reference surface.

[0096] Step S230: Input the pixel coordinates in the two-dimensional image into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface.

[0097] Step S240: The difference between the depth information and the corresponding height value of the depth map is used as the corrected depth information.

[0098] For more complex PCB substrate conditions, a backpropagation (BP) neural network can be used to fit the relationship between pixel coordinates (i, j) and depth information z in a two-dimensional image. Specifically, using the depth map pixels of the substrate area segmented in step S210, a three-layer BP neural network is trained to establish the mapping relationship between pixel coordinates (i, j) and depth information z, which is the model of the PCB substrate. In the specific training process, methods such as... Figure 6 The BP neural network model shown downsamples the pixel region to accelerate the training process. The input sample of the BP neural network model is the pixel coordinates (i, j) of the segmented substrate region, and the output sample is the corresponding height value. Through backpropagation of error, the connection weights and neuron thresholds of each layer in the BP neural network model are corrected layer by layer, so that the error gradually decreases. The training is repeated until the error meets the accuracy requirements.

[0099] By inputting the pixel coordinates from the 2D image into a trained BP neural network model, the height value corresponding to each pixel coordinate on a preset reference surface can be obtained. The correction result for each point in the depth map is the actual depth h'(i,j) of that point, which can be directly corrected using the height value obtained from the BP neural network.

[0100] h'(i,j)=h(i,j)-z(i,j)

[0101] Where h(i,j) is the depth information measured at point (i,j), and z(i,j) is the height value obtained by the BP neural network model at point (i,j).

[0102] For complex PCB substrates, a BP neural network model can be trained to obtain the height value for each point and make corrections based on the height value. This effectively corrects the depth map deviation caused by substrate tilting or bending, thereby reducing false detections.

[0103] Figure 7 This is a schematic diagram of a PCB image correction device provided in an embodiment of the present invention. (Reference) Figure 7 The PCB image correction device includes: a substrate area determination unit 310, a coordinate data confirmation unit 320, a coordinate data fitting unit 330, and a depth map correction unit 340.

[0104] The substrate region determination unit 310 is used to determine the corresponding substrate region based on the depth map and two-dimensional map of the PCB under test; the coordinate data confirmation unit 320 is used to confirm the three-dimensional coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the two-dimensional map; the coordinate data fitting unit 330 is used to randomly select a set number of points from the three-dimensional coordinate data for fitting to obtain a reference plane; and the depth map correction unit 340 is used to correct the depth of each point in the depth map based on the reference plane and with a preset reference plane as a reference to obtain a corrected depth map.

[0105] Based on the above embodiments, the coordinate data fitting unit 330 includes:

[0106] The initial fitting module is used to randomly select a preset number of points from the three-dimensional coordinate data for fitting, so as to obtain at least two initial reference surfaces;

[0107] The distance calculation module is used to calculate the distance from each point to each initial reference surface based on the three-dimensional coordinate data;

[0108] The effective point statistics module is used to count the effective points corresponding to each initial reference surface. The effective points are those whose distance from the initial reference surface is less than a preset distance threshold.

[0109] The reference plane confirmation module is used to select the initial reference plane with the most valid points as the reference plane of the substrate region.

[0110] Based on the above embodiments, the substrate region determination unit 310 includes:

[0111] The first preprocessing module is used to preprocess the depth map of the PCB to be tested to obtain the first substrate area;

[0112] The second preprocessing module is used to preprocess the two-dimensional image of the PCB to be tested to obtain the second substrate area;

[0113] The intersection confirmation module is used to confirm the substrate region based on the intersection region of the first substrate region and the second substrate region.

[0114] Based on the above embodiments, the first preprocessing module is specifically used to segment the depth map of the PCB to be tested based on a preset height threshold, extract the part of the depth map that is lower than the height threshold, and obtain the first substrate area.

[0115] Based on the above embodiments, the second preprocessing module includes:

[0116] The grayscale statistics submodule is used to perform grayscale histogram statistics on the three color channels of the two-dimensional image respectively;

[0117] The pending area confirmation submodule is used to confirm the pending mask area corresponding to each color channel based on the preset grayscale range value.

[0118] The intersection region confirmation submodule is used to obtain the second substrate region based on the intersection of the undetermined mask regions corresponding to each color channel.

[0119] Based on the above embodiments, the two-dimensional image is obtained by acquiring it through a telecentric lens;

[0120] The coordinate data confirmation unit is specifically used to perform a proportional conversion of the pixel coordinates of the two-dimensional image based on the parameters of the telecentric lens to obtain the x and y coordinates in the three-dimensional coordinate data, obtain the depth information of the substrate region corresponding to the depth image as the z coordinate, and use the corresponding x, y, and z coordinates as the three-dimensional coordinate data of the substrate region.

[0121] Based on the above embodiments, the reference surface is a plane or a curved surface. The spatial equation of the plane is k0x + k1y + z + k2 = 0, and the spatial equation of the curved surface is k3x. 2 +k4y 2 +k5xy+k6x+k7y+k8+z=0;

[0122] The depth map correction unit 340 includes:

[0123] The first correction module is used to correct points in the depth map using the following first depth correction formula if the reference plane is a plane:

[0124]

[0125] The second correction module is used to correct points in the depth map using the following second depth correction formula if the reference surface is a curved surface:

[0126]

[0127] Where (i,j) are the pixel coordinates corresponding to the midpoint (x,y) in the three-dimensional coordinate data, h'(i,j) are the correction depth corresponding to the midpoint (x,y) in the three-dimensional coordinate data, h(i,j) are the measurement depth corresponding to the midpoint (x,y) in the three-dimensional coordinate data, mi = x, mj = y, and m is the object resolution of the camera that acquires the two-dimensional image.

[0128] The PCB image correction device provided in this embodiment of the invention is included in a terminal device and can be used to execute the PCB image correction method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0129] It is worth noting that in the embodiments of the PCB image correction device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0130] Figure 8 This is a schematic diagram of a PCB image correction device provided in an embodiment of the present invention. (Reference) Figure 8 The PCB image correction device includes: a substrate area determination unit 410, a model training unit 420, a height value output unit 430, and a depth information correction unit 440.

[0131] The substrate area determination unit is used to determine the corresponding substrate area based on the depth map and two-dimensional map of the PCB under test.

[0132] The model training unit is used to train a BP neural network model by using the depth information associated with the depth map and the substrate region, as well as the pixel coordinates associated with the substrate region in the two-dimensional map, as samples. The BP neural network model is used to output the height value of the pixel coordinates corresponding to a preset reference surface.

[0133] The height value output unit is used to input the pixel coordinates in the two-dimensional image into the BP neural network model to obtain the height value of each pixel coordinate corresponding to the preset reference surface;

[0134] The depth information correction unit is used to take the difference between the depth information and the corresponding height value of the depth map as the corrected depth information.

[0135] The PCB image correction device provided in this embodiment of the invention is included in a terminal device and can be used to execute the PCB image correction method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0136] It is worth noting that in the embodiments of the PCB image correction device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0137] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present invention. Figure 9 As shown, the electronic device includes a processor 510, a memory 520, an input device 530, an output device 540, and a communication device 550; the number of processors 510 in the electronic device can be one or more. Figure 9Taking a processor 510 as an example; the processor 510, memory 520, input device 530, output device 540, and communication device 550 in the electronic device can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0138] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the PCB image correction method in this embodiment of the invention (e.g., the substrate area determination unit 310, coordinate data confirmation unit 320, coordinate data fitting unit 330, and depth map correction unit 340 in the PCB image correction device). Of course, the program instructions / modules that implement the functions of different electronic devices are located in the corresponding electronic devices. The processor 310 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 320, thereby implementing the aforementioned PCB image correction method.

[0139] The memory 320 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 320 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include memory remotely located relative to the processor 310, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0140] Input device 330 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 340 may include display devices such as a display screen.

[0141] The aforementioned electronic device includes a PCB image correction device, which can be used to perform any PCB image correction method and has corresponding functions and beneficial effects.

[0142] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform relevant operations in the PCB image correction method provided in any embodiment of this application, and have corresponding functions and beneficial effects.

[0143] Those skilled in the art will understand that embodiments of this application may be provided as methods, systems, or computer program products.

[0144] Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0148] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A PCB image correction method, characterized in that, include: Identify the corresponding substrate area based on the depth map and 2D map of the PCB under test; Based on the depth information of the depth map and the coordinates of the two-dimensional map, the pixel coordinates are converted to physical coordinates to confirm the three-dimensional coordinate data of the substrate area. A reference surface is obtained by randomly selecting a set number of points from the three-dimensional coordinate data and fitting them. Based on the reference plane, and with a preset reference plane as a reference, the depth of each point in the depth map is corrected to obtain a corrected depth map; Wherein, the reference surface is a plane or a curved surface, and the spatial equation of the plane is: The spatial equation of the surface is ; The step of correcting the depth of each point in the depth map based on the reference plane and using a preset reference plane as a reference to obtain a corrected depth map includes: If the reference plane is a plane, the points in the depth map are corrected using the following first depth correction formula: If the reference surface is a curved surface, the points in the depth map are corrected using the following second depth correction formula: in, The midpoint of the three-dimensional coordinate data The corresponding pixel coordinates The midpoint of the three-dimensional coordinate data Corresponding correction depth The midpoint of the three-dimensional coordinate data The corresponding measurement depth, = , = , The object-side resolution of the camera that acquires the two-dimensional image; , and The coefficients of the spatial equation of the plane are denoted as . , , , , and are the coefficients of the spatial equation of the surface.

2. The method according to claim 1, characterized in that, The step of randomly selecting a predetermined number of points from the three-dimensional coordinate data for fitting to obtain a reference surface includes: A predetermined number of points are randomly selected from the three-dimensional coordinate data for fitting to obtain at least two initial reference surfaces; Calculate the distance from each point to each initial reference plane based on the three-dimensional coordinate data; Count the valid points corresponding to each initial reference plane. The valid points are those whose distance from the initial reference plane is less than a preset distance threshold. The initial reference plane with the most effective points is used as the reference plane for the substrate region.

3. The method according to claim 1, characterized in that, The step of identifying the corresponding substrate area based on the depth map and 2D map of the PCB under test includes: The depth map of the PCB under test is preprocessed to obtain the first substrate region; The two-dimensional image of the PCB to be tested is preprocessed to obtain the second substrate area; The substrate region is identified based on the intersection of the first substrate region and the second substrate region.

4. The method according to claim 3, characterized in that, The depth map of the PCB under test is preprocessed to obtain the first substrate region, specifically as follows: Based on a preset height threshold, the depth map of the PCB under test is segmented, and the portion of the depth map below the height threshold is extracted to obtain the first substrate region.

5. The method according to claim 3, characterized in that, The 2D image of the PCB to be tested is preprocessed to obtain the second substrate region, specifically: Perform grayscale histogram statistics on the three color channels of the two-dimensional image respectively; Based on the preset grayscale range values, the undetermined mask area corresponding to each color channel is identified; The second substrate region is obtained by the intersection of the undetermined mask regions corresponding to each color channel.

6. The method according to claim 1, characterized in that, The two-dimensional image was acquired using a telecentric lens; The process of determining the three-dimensional coordinate data of the substrate region based on the depth information of the depth map and the coordinates of the two-dimensional map specifically involves: Based on the parameters of the telecentric lens, the pixel coordinates of the two-dimensional image are proportionally converted to obtain the x and y coordinates in the three-dimensional coordinate data. The depth information of the substrate region corresponding to the depth image is obtained as the z coordinate, and the corresponding x, y, and z coordinates are used as the three-dimensional coordinate data of the substrate region.

7. A PCB image correction device, characterized in that, include: The substrate area determination unit is used to determine the corresponding substrate area based on the depth map and two-dimensional map of the PCB under test. The coordinate data confirmation unit is used to convert pixel coordinates to physical coordinates based on the depth information of the depth map and the coordinates of the two-dimensional map, so as to confirm the three-dimensional coordinate data of the substrate area. The coordinate data fitting unit is used to randomly select a set number of points from the three-dimensional coordinate data for fitting to obtain a reference surface; The depth map correction unit is used to correct the depth of each point in the depth map based on the reference plane and with a preset reference plane as a reference, so as to obtain a corrected depth map. Wherein, the reference surface is a plane or a curved surface, and the spatial equation of the plane is: The spatial equation of the surface is ; The step of correcting the depth of each point in the depth map based on the reference plane and using a preset reference plane as a reference to obtain a corrected depth map includes: If the reference plane is a plane, the points in the depth map are corrected using the following first depth correction formula: If the reference surface is a curved surface, the points in the depth map are corrected using the following second depth correction formula: in, The midpoint of the three-dimensional coordinate data The corresponding pixel coordinates The midpoint of the three-dimensional coordinate data Corresponding correction depth The midpoint of the three-dimensional coordinate data The corresponding measurement depth, = , = , The object-side resolution of the camera that acquires the two-dimensional image; , and The coefficients of the spatial equation of the plane are denoted as . , , , , and are the coefficients of the spatial equation of the surface.

8. A terminal device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the terminal device implements the PCB image correction method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the PCB image correction method as described in any one of claims 1-6.

Images