Data recovery method

Abstract

The invention discloses a data restoration method, and the method comprises the steps: carrying out the segmentation processing of an RGBW image according to the point cloud data of a to-be-detected workpiece, the RGBW image, and a preset segmentation algorithm, and determining a plurality of target regions and data missing regions; fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; in the converted point cloud data, determining point cloud data around each point in the data missing area, and determining an effective value of each point; according to the position relation between the data missing area and the target areas and the angle value and the effective value of each point in the data missing area, the data missing area is repaired, and point cloud data of the repaired workpiece to be detected are obtained. Therefore, the corresponding repair modes are collected for the data-lacking areas located in the different target areas, and the point cloud data of the repaired to-be-measured workpiece can better restore the real height of the to-be-measured workpiece.

Description

Technical Field

[0001] This application relates to the technical field of three-dimensional automated optical inspection equipment, and more specifically, to a data repair method. Background Technology

[0002] The workpiece under test is equipped with electronic components (e.g., resistors, capacitors, etc.). The point cloud data of the workpiece under test is a set of height points of the components on the workpiece under test. Defects of the components, such as warping and poor soldering, can be detected through this point cloud data.

[0003] However, since the materials of components are usually highly reflective, light-absorbing, transparent or semi-transparent, the point cloud data of the workpiece under test may be lost. Using the point cloud data of the workpiece under test that contains the lost data cannot restore its true height, which has a certain impact on the defects of the detection component. Summary of the Invention

[0004] In view of the above problems, this application proposes a data repair method to solve the above problems.

[0005] This application provides a data repair method, which includes: acquiring point cloud data and an RGBW image of a workpiece to be tested; segmenting the RGBW image according to the point cloud data, the RGBW image, and a preset segmentation algorithm to determine multiple target regions and data-missing regions; fitting a zero plane according to the point cloud data and the least squares method, and converting the point cloud data to the zero plane to obtain converted point cloud data; determining the point cloud data surrounding each point in the converted point cloud data and determining the effective value of each point; repairing the data-missing regions according to the positional relationship between the data-missing regions and multiple target regions, the angle value of each point in the data-missing regions, and the effective value, to obtain repaired point cloud data of the workpiece to be tested. Therefore, by identifying the data-missing regions contained in different target regions, targeted repair methods can be adopted to make the repaired point cloud data of the workpiece to be tested more complete, thereby better restoring the true height of the workpiece to be tested and reducing the impact on the detection and other related operations of the workpiece to be tested. Attached Figure Description

[0006] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments and drawings obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0007] Figure 1A schematic diagram of the structure of a three-dimensional automated optical inspection device according to an embodiment of this application is shown.

[0008] Figure 2 A flowchart illustrating a data repair method provided in an embodiment of this application is shown.

[0009] Figure 3 A schematic diagram of the structure of an RGBW image provided in an embodiment of this application is shown.

[0010] Figure 4 A schematic diagram of the structure of a data repair device provided in an embodiment of this application is shown.

[0011] Figure 5 A schematic diagram of the structure of a three-dimensional automated optical inspection device provided in an embodiment of this application is shown.

[0012] Figure 6 This illustration shows a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0014] The workpiece under test is equipped with electronic components (e.g., resistors, capacitors, etc.). The point cloud data of the workpiece under test is a set of height points of the components on the workpiece under test. Defects of the components, such as warping and poor soldering, can be detected through this point cloud data.

[0015] However, since the materials of components are usually highly reflective, light-absorbing, transparent or semi-transparent, the point cloud data of the workpiece under test may be lost. Using the point cloud data of the workpiece under test that contains the lost data cannot restore its true height, which has a certain impact on the defects of the detection component.

[0016] To address the aforementioned issues, this application provides a data repair method, comprising: acquiring point cloud data and an RGBW image of the workpiece to be tested; segmenting the RGBW image based on the point cloud data, the RGBW image, and a preset segmentation algorithm to determine multiple target regions and data-missing regions; fitting a zero plane based on the point cloud data and the least squares method, and converting the point cloud data to the zero plane to obtain converted point cloud data; determining the point cloud data surrounding each point in the converted point cloud data, and determining the valid value of each point; repairing the data-missing regions based on the positional relationship between the data-missing regions and the multiple target regions, the angle value of each point in the data-missing regions, and the valid value, to obtain repaired point cloud data of the workpiece to be tested.

[0017] Therefore, by identifying the data-missing areas contained in different target regions, targeted repair methods can be adopted to make the point cloud data of the workpiece under test more complete after repair, thereby better restoring the true height of the workpiece under test and reducing the impact on the detection and other related operations of the workpiece under test.

[0018] The following describes the application environment of the data repair method provided in the embodiments of the present invention.

[0019] Please see Figure 1 , Figure 1 A schematic diagram of the structure of a three-dimensional automated optical inspection device according to an embodiment of this application is shown, as follows: Figure 1 As shown, the three-dimensional automatic optical inspection device 100 includes a light source, an optomechanical system 110, and an image acquisition module 120.

[0020] The light source is used to illuminate the workpiece to be tested located at the detection position of the three-dimensional automatic optical inspection equipment 100; the optomechanical system 110 is used to scan the workpiece to be tested in order to collect the point cloud data of the workpiece; and the image acquisition module 120 is used to acquire the RGBW image of the workpiece to be tested.

[0021] In some implementations, the three-dimensional automated optical inspection device 100 can be a 3D AOI.

[0022] In some implementations, the light source may include RGB light sources and W light sources.

[0023] In some implementations, the image acquisition module 120 can be a camera.

[0024] In embodiments of this application, the three-dimensional automated optical inspection device 100 may include a preset number of optical engines 110. For example, the three-dimensional automated optical inspection device 100 may include four optical engines 110. As another example, the three-dimensional automated optical inspection device 100 may include eight optical engines 110. It is understood that this application does not limit the specific number of optical engines 110.

[0025] In the embodiments of this application, the three-dimensional automatic optical inspection device 100 may include four optical engines 110, and the four optical engines 110 are respectively located on the upper side, lower side, left side and right side of the workpiece to be tested, so as to obtain relatively complete point cloud data of the workpiece to be tested.

[0026] By properly configuring the optical engine 110, the problem of incomplete point cloud data in the shaded area due to the excessive height of the workpiece under test can be effectively solved. Multiple optical engines 110 located in different directions scan the workpiece under test, and the collected point cloud data is fused to obtain the final point cloud data of the workpiece.

[0027] In some implementations, the workpiece to be tested can be a PCB board, on which components are also disposed. The workpiece to be tested can be placed at the detection position of the three-dimensional automated optical inspection equipment 100, so that the point cloud data corresponding to the workpiece to be tested can be acquired by the optomechanical system 110, and the RGBW image corresponding to the workpiece to be tested can be acquired by the image acquisition module 120.

[0028] In some implementations, the point cloud data of the workpiece under test can be a set of height values ​​for the workpiece, i.e., the height values ​​of the components installed on the workpiece. This point cloud data can be used to detect defects in the components, such as warping or poor soldering.

[0029] However, when the material of the components on the workpiece under test is highly reflective, absorbent, transparent, or translucent, the point cloud data scanned by the optomechanical 110 may be lost. Therefore, the true height of the components cannot be reconstructed from this point cloud data, which affects related operations, such as component inspection. Thus, it is necessary to repair the point cloud data of the scanned workpiece. Specifically: Please see Figure 2 , Figure 2 A flowchart illustrating a data repair method provided in an embodiment of this application is shown, which can be applied to the aforementioned three-dimensional automated optical inspection equipment. For example... Figure 2 As shown, the method may include steps 210 to 220.

[0030] In step 210, the point cloud data and RGBW image of the workpiece to be tested are acquired.

[0031] The three-dimensional automatic optical inspection equipment acquires point cloud data of the workpiece under test through multiple optomechanical systems, and fuses the point cloud data acquired by the multiple optomechanical systems to obtain the point cloud data of the workpiece under test.

[0032] The three-dimensional automatic optical inspection equipment acquires images of the workpiece under test through an image acquisition module to obtain the RGBW image of the workpiece under test.

[0033] In subsequent steps, based on the height values ​​of the components included in the point cloud data of the workpiece under test and the color information included in the RGBW image, the RGBW image is segmented into multiple target regions and data missing regions included in each of the target regions. Specific repair methods are then applied to the data missing regions located in different target regions to make the final repaired point cloud data of the workpiece under test more complete. This allows for a better restoration of the true height of the workpiece under test, reducing the impact on the inspection and other related operations.

[0034] In step 220, the RGBW image is segmented based on point cloud data, RGBW image and preset segmentation algorithm to determine multiple target regions and data missing regions.

[0035] A preset segmentation algorithm can segment an RGBW image based on the height data included in the point cloud data and the color information included in the RGBW image, thereby identifying multiple target regions on the RGBW image and the data missing regions included in each of the target regions. In some embodiments, the preset segmentation algorithm can be a clustering segmentation algorithm. It is understood that this application does not limit the specific algorithm used for the preset segmentation algorithm.

[0036] In some implementations, multiple target areas may include a substrate area, a body area, and a solder paste area on the PCB board. That is, the missing data area may be located on the substrate area, the body area, or the solder paste area. By processing the point cloud data within the missing data areas on the substrate area, the body area, and the solder paste area, the point cloud data of the repaired workpiece is made more complete, thereby better restoring the true height of the workpiece and reducing the need for inspection and related operations.

[0037] For example, please refer to Figure 3 , Figure 3 This illustration shows a structural diagram of an RGBW image provided in an embodiment of this application, as shown below. Figure 3 As shown, the RGBW image includes substrate region A, body region B, solder paste region C, and data missing region D.

[0038] Specifically, in some implementations, the step "segmenting the RGBW image based on point cloud data, the RGBW image, and a preset segmentation algorithm to determine multiple target regions and data-missing regions" may include the following steps: (1) Based on the point cloud data, RGBW image and preset segmentation algorithm, the RGBW image is segmented to generate a height area distribution map containing multiple regions.

[0039] (2) The region with the largest area and the lowest height value among the multiple regions is determined as the substrate region.

[0040] (3) The regions in which the color information is a preset color are identified as solder paste regions.

[0041] (4) The area with the largest height value among the multiple regions is smaller than the substrate area, larger than the solder paste area, and is the highest.

[0042] (5) Areas with height values ​​within a preset height range or areas smaller than a preset value are identified as data missing areas.

[0043] In some implementations, the three-dimensional automated optical inspection equipment determines the area of ​​each region based on the number of pixels included in each of the multiple regions. That is, it determines the areas of the substrate region, the body region, the solder paste region, and the data-missing region, respectively.

[0044] In some implementations, the three-dimensional automated optical inspection equipment determines the height value of each region based on the average height values ​​of the point cloud data included in each of the multiple regions. Specifically, it determines the height values ​​of the substrate region, the body region, the solder paste region, and the data-missing region.

[0045] After determining the area and height values ​​of each region in the height-area distribution map, the 3D automated optical inspection equipment determines the specific type of each region based on these values. Specifically, it identifies each region as a substrate area, body area, solder paste area, or area with missing data.

[0046] In some implementations, the preset color can be blue. The three-dimensional automated optical inspection equipment identifies areas with blue color information as solder paste areas on the RGBW image.

[0047] In some implementations, the preset height range can be an interval of (-7000, 30000). It is understood that this application does not limit the specific interval of the preset height range. The three-dimensional automatic optical inspection equipment, based on the collected point cloud data of the workpiece to be tested, determines the area enclosed by point cloud data with height data within the interval (-7000, 30000) as the data missing area.

[0048] In some implementations, the preset value can be 100. It is understood that this application does not limit the specific value of the preset value. After determining the area of ​​multiple regions, the three-dimensional automated optical inspection equipment identifies regions with an area less than 100 as data-missing regions.

[0049] In other words, the 3D automatic optical inspection equipment identifies the area enclosed by point cloud data with height data in the range of (-7000, 30000) as the data missing area, or the area with an area less than 100 as the data missing area.

[0050] The three-dimensional automated optical inspection equipment identifies the region with the largest area and the lowest height value of the point cloud data it contains as the substrate region; and identifies the region with the second largest area and the highest height value as the body region.

[0051] The three-dimensional automated optical inspection equipment, through the aforementioned segmentation method, identifies the substrate area, body area, solder paste area, and data-deficient area in the height-area distribution map. It is understandable that by identifying the data-deficient area in the height-area distribution map and then uniformly repairing all data-deficient areas, a relatively complete repaired point cloud data of the workpiece under test can be obtained. However, this application, by separately identifying the substrate area, body area, and solder paste area in the height-area distribution map, and then determining which area of ​​the substrate area, body area, and solder paste area the data-deficient area is located in, and then collecting targeted repair methods based on the location of the data-deficient area, aims to make the repaired point cloud data closer to the most realistic condition of the workpiece under test, improving the accuracy of the repaired point cloud data, and thus better reproducing the workpiece under test. Specifically: In step 230, the zero plane is fitted based on the point cloud data and the least squares method, and the point cloud data is transformed to the zero plane to obtain the transformed point cloud data.

[0052] Before a 3D automated optical inspection device can determine which region (substrate, body, or solder paste) the missing data area is located in, it is necessary to unify the point cloud data of the workpiece under test into a common coordinate system.

[0053] The 3D automated optical inspection equipment substitutes the point cloud data of the workpiece under test into the least squares method to find the best-fit plane, i.e., the zero plane, in the point cloud data of the workpiece under test. After determining the zero plane corresponding to the point cloud data, the point cloud data is then transformed to the zero plane to obtain the transformed point cloud data, thereby unifying the point cloud data of the workpiece under test into the same coordinate system.

[0054] To reduce the impact of outliers in the collected point cloud data of the workpiece under test, and thus minimize errors in fitting the zero plane to the point cloud data, in some implementations, the 3D automated optical inspection equipment sorts the point cloud data of the workpiece under test according to its corresponding height values ​​from high to low, obtaining a sorted point cloud dataset. Then, it removes the top 10% and bottom 10% of the point cloud data from the sorted dataset, obtaining the final point cloud dataset. Finally, it fits the zero plane using the least squares method based on the final point cloud dataset and transforms the final point cloud dataset to the zero plane, thus unifying the coordinate system of the final point cloud data.

[0055] The 3D automated optical inspection equipment uses point cloud data with a unified coordinate system to determine the valid value of each pixel in the data missing area, so that subsequent steps can be corrected based on the valid values ​​of the pixels. Specifically: In step 240, in the transformed point cloud data, the point cloud data surrounding each point in the data missing region is identified, and the valid value of each point is determined.

[0056] The three-dimensional automated optical inspection equipment searches for the closest valid data to each point in the data missing area based on the surrounding area of ​​each point. That is, it identifies and records the points in the substrate area, body area or solder paste area that are closest to each point in the data missing area to determine the valid value of each point.

[0057] Specifically, in some implementations, the step "in the converted point cloud data, identify the point cloud data surrounding each point in the data missing region and determine the valid value for each point" may include the following steps: (1) Determine the location information of each point in the data missing area.

[0058] (2) Based on the location information of each point, determine the effective data in the eight neighborhood directions of each point; the effective data is the point cloud data included in the target area.

[0059] (3) Determine the effective value of each point based on the valid data.

[0060] After acquiring the position information of each pixel in the data-missing area, the 3D automated optical inspection equipment searches in eight neighboring directions based on the position information of each pixel. It finds the valid data (i.e., point cloud data in the substrate area, body area, or solder paste area) for each pixel in each direction, and records the coordinates and height values ​​of the valid positions to determine the valid value of each point. Further: In step 250, the missing data area is repaired based on the positional relationship between the missing data area and multiple target areas, the angle value and valid value of each point in the missing data area, and the repaired point cloud data of the workpiece to be tested is obtained.

[0061] The positional relationship between the missing data area and multiple target areas, i.e., whether the missing data area is located in the substrate area, the body area, or the solder paste area, is used to collect targeted repair methods based on which area the missing data area is located in.

[0062] The angle value of each point in the missing data region can be determined using the angle compensation map corresponding to the RGBW image. Specifically, in some implementations, this data repair method may include the following steps: (1) Calculate the angle value of the pixel based on the R, G, B and W values ​​of the pixel in the RGBW image to determine the angle value of the pixel.

[0063] (2) Determine the angle compensation map based on the angle value of the pixel.

[0064] The three-dimensional automatic optical inspection equipment determines the angle value corresponding to each pixel in the RGBW image based on the R, G, B or W values, in order to determine the angle compensation map of the workpiece to be tested.

[0065] Specifically, in some implementations, the step "calculate the angle value of the pixel based on the R, G, B and W values ​​corresponding to the pixel in the RGBW image to determine the angle value of the pixel" may include the step of: determining the angle value of the pixel based on the R, G, B and W values ​​of the pixel and the angle calculation equation.

[0066] The equation for calculating the angle can be expressed as: in," " represents the angle value of the pixel." "W value of pixel", " is the R value of the pixel, " is the G value of the pixel, " " is the B value of the pixel.

[0067] The three-dimensional automatic optical inspection equipment uses the aforementioned angle calculation equation to determine the angle value corresponding to each pixel based on the specific values ​​of the R, G, B, or W values ​​corresponding to each pixel, thereby determining the angle value of each point in the data missing area.

[0068] After determining the angle value corresponding to each pixel in the RGBW image, the 3D automated optical inspection equipment uses a seed filling algorithm to repair each missing data region, thereby obtaining the point cloud data corresponding to each missing data region. Specifically: In some implementations, the step "repairing the missing data region based on the positional relationship between the missing data region and multiple target regions, the angle value and valid value of each point in the missing data region, to obtain the repaired point cloud data of the workpiece to be tested" may include the following steps: (1) If the missing data area is located in the substrate area, the point cloud data after repair of each point is determined according to the comparison relationship between the angle value of each point and the first preset value.

[0069] (2) If the missing data area is located in the middle area of ​​the body area or the middle area of ​​the solder paste area, then the point cloud data after repair of each point is determined according to the angle value of each point.

[0070] (3) If the missing data area is located in other areas of the body area or other areas of the solder paste area, then the point cloud data after repair of each point is determined according to the angle value and valid value of each point.

[0071] In some implementations, the step "determining the repaired point cloud data for each point based on the comparison between the angle value of each point and the first preset value" may include the following steps: (1) Set the repaired point cloud data corresponding to the points in the data missing area with an angle value less than the first preset value as the preset value.

[0072] (2) Set the repaired point cloud data corresponding to the points in the data missing area whose angle value is greater than or equal to the first preset value as the mean of its effective value.

[0073] In some implementations, the first preset value can be 0. It is understood that this application does not limit the specific value of the first preset value.

[0074] In some implementations, the preset value can be 0. It is understood that this application does not limit the specific value of the preset value.

[0075] If the angle value corresponding to a pixel in the substrate area is less than 0, it indicates that these data points are outliers or noise. In this case, the 3D automated optical inspection equipment sets the point cloud data corresponding to that pixel to 0. If the angle value corresponding to a pixel in the substrate area is greater than or equal to 0, the 3D automated optical inspection equipment sets the point cloud data corresponding to that pixel to the average of the valid values ​​corresponding to that pixel. This method, by using valid values ​​around the data point to fill in missing values ​​or correct possible errors, helps to ensure the continuity and integrity of the data.

[0076] Because the substrate area has a high surface flatness or low defect rate, the three-dimensional automatic optical inspection equipment can use the above-mentioned repair method to make the data points in the substrate area transition smoothly and effectively maintain the structural characteristics of the substrate area, so as to complete the repair of the point cloud data corresponding to the substrate area.

[0077] The middle region of the body region or the middle region of the solder paste region can be preset. For example, in the body region and the solder paste region, the region that overlaps with the specified region of the RGBW image is determined as the middle region of the body region and the middle region of the solder paste region.

[0078] If the missing data area is located in the middle of the body area or the middle of the solder paste area, the point cloud data corresponding to each point is estimated based on the angle value corresponding to each point in the missing data area.

[0079] Specifically, the middle area of ​​the solder paste region is defined as the starting point, and the middle area of ​​the body region is defined as the ending point. Each point can be filled using the angle value corresponding to that point. It is understood that this application does not limit the starting and ending points; for example, filling can begin from one end of the middle area of ​​the solder paste region and proceed towards the other end.

[0080] If the missing data area is located in other areas of the body area or other areas of the solder paste area, that is, not in the middle area of ​​the body area or the middle area of ​​the solder paste area, then the point cloud data after repair for each point is determined based on whether the angle value of each point is consistent with the angle value corresponding to the valid value.

[0081] Furthermore, in some implementations, the step "if the missing data area is located in another area of ​​the body area or another area of ​​the solder paste area, then determine the repaired point cloud data for each point based on the angle value and valid value of each point" may include the following steps: (1) If the angle value is consistent with the angle value corresponding to its corresponding effective value, then the point cloud data after the point is repaired is determined to be the mean of its corresponding effective value.

[0082] (2) If the angle value is inconsistent with the angle value corresponding to its effective value, the point cloud data after the point is repaired shall be determined as its effective value.

[0083] The 3D automatic optical inspection equipment determines whether the angle value corresponding to the effective value of each data point in the missing data area, located in other areas of the body area or other areas of the solder paste area, is consistent with the angle value corresponding to it in the angle compensation map. If they are consistent, it means that the angle value corresponding to the effective value of the data point has similar directionality or angle characteristics to the angle value corresponding to it in the angle compensation map. By filling it with the mean of the effective value corresponding to the data point, the filled data can be smoother and more natural, while maintaining the original angle characteristics.

[0084] If there is a discrepancy, it indicates that the angle value corresponding to the valid value of the data point has a different directionality or angular characteristic than the angle value corresponding to it in the angle compensation map. This may indicate that the area formed by the data point and its associated data points has a unique geometric structure, surface variation, or other physical phenomenon. In this case, using the mean of the valid values ​​corresponding to the data point to fill it may introduce errors because this ignores information about local variations. Therefore, in this situation, using the valid values ​​corresponding to the data points to fill it is better to reflect the actual geometric shape or surface properties, rather than simply "smoothing out" the differences.

[0085] It is worth noting that, in the embodiments of this application, the three-dimensional automatic optical inspection device prioritizes filling the data missing regions with the smallest height difference between the valid values ​​of the included data points. By prioritizing the repair of the data missing regions with the smallest height difference of the valid values ​​and marking them as valid data, the range of valid data can be expanded, allowing other data points to fill the corresponding point cloud data more quickly.

[0086] After the 3D automated optical inspection equipment repairs the point cloud data to be inspected using the above method, in order to further detect whether there are still missing data points in the repaired point cloud data, and to fill in the missing data points in a timely manner, the data repair method may further include the following steps in some embodiments: (1) Update the point cloud data to the repaired point cloud data; (2) Based on the updated point cloud data, RGBW image and preset segmentation algorithm, the RGBW image is segmented to determine whether there are missing data regions.

[0087] (2) If there are missing data areas, the following steps are performed: based on the point cloud data and the least squares method, fit the zero plane and convert the point cloud data to the zero plane to obtain the converted point cloud data.

[0088] The three-dimensional automatic optical inspection equipment performs a second round of segmentation processing on the RGBW image based on the repaired point cloud data of the workpiece under test, the RGBW image, and the preset segmentation algorithm to determine whether there is any missing point cloud data in the current workpiece under test. If there is missing point cloud data, the missing point cloud data is repaired according to the above point cloud data repair method until there is no missing point cloud data.

[0089] In other words, in this application, the point cloud data of the workpiece under test can be repaired in one round, or at least in two rounds. This makes the final repaired point cloud data of the workpiece under test more complete, thereby better restoring the true height of the workpiece and reducing the accuracy of related operations such as workpiece inspection.

[0090] Please see Figure 4 , Figure 4 This illustration shows a schematic diagram of a data repair device provided in an embodiment of this application, applied to the aforementioned three-dimensional automated optical inspection equipment. The data repair device 300 includes: an acquisition module 310, a first execution module 320, a second execution module 330, a determination module 340, and a third execution module 350. Specifically: The acquisition module 310 is used to acquire point cloud data and RGBW images of the workpiece to be tested.

[0091] The first execution module 320 is used to segment the RGBW image based on point cloud data, RGBW image and preset segmentation algorithm to determine multiple target regions and data missing regions.

[0092] The second execution module 330 is used to fit a zero plane based on the point cloud data and the least squares method, and to transform the point cloud data to the zero plane to obtain the transformed point cloud data.

[0093] The determination module 340 is used to determine the point cloud data around each point in the data missing region in the transformed point cloud data, and to determine the valid value of each point.

[0094] The third execution module 350 is used to repair the data missing area based on the positional relationship between the data missing area and multiple target areas, the angle value and valid value of each point in the data missing area, and to obtain the point cloud data of the workpiece under test after repair.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0096] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0097] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0098] Please see Figure 5 , Figure 5 The diagram shows a structural schematic of a three-dimensional automated optical inspection device provided in an embodiment of this application. The three-dimensional automated optical inspection device 400 in this application may include one or more of the following components: a processor 410, a memory 420, and one or more application programs. The one or more application programs may be stored in the memory 420 and configured to be executed by one or more processors 410. The one or more programs are configured to perform the data repair method as described in the foregoing method embodiments.

[0099] The processor 410 may include one or more processing cores. The processor 410 connects to various parts within the three-dimensional automated optical inspection device 400 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 420, and by calling data stored in the memory 420. Optionally, the processor 410 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 410 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 410 and may be implemented separately using a communication chip.

[0100] The memory 420 may include random access memory (RAM) or read-only memory (ROM). The memory 420 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 420 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created during the use of the three-dimensional automated optical inspection device 400.

[0101] Please see Figure 6 , Figure 6 The diagram shows a computer-readable storage medium 500 provided in an embodiment of this application. The computer-readable storage medium 500 stores program code, which can be called by a processor to execute the data repair method described in the above method embodiment.

[0102] The computer-readable storage medium 500 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any of the method steps described above. This program code can be read from or written to one or more computer program devices. The program code 510 may, for example, be compressed in a suitable form.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data repair method characterized by, The method is applied to a three-dimensional automatic optical inspection device, and comprises the following steps: Obtaining point cloud data and an RGBW image of a workpiece to be measured; Segmenting the RGBW image according to the point cloud data, the RGBW image and a preset segmentation algorithm to determine a plurality of target regions and a data missing region; Fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; In the converted point cloud data, determining point cloud data located around each point in the data missing region and determining the effective value of each point; Repairing the data missing region according to the positional relationship between the data missing region and the plurality of target regions, the angle value of each point in the data missing region and the effective value to obtain repaired point cloud data of the workpiece to be measured.

2. The data repair method of claim 1, wherein, The method further comprises the following steps: Updating the point cloud data to the repaired point cloud data; Segmenting the RGBW image according to the updated point cloud data, the RGBW image and the preset segmentation algorithm to determine whether the data missing region exists; If the data missing region exists, fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data.

3. The data repair method of claim 1, wherein, The plurality of target regions comprise a substrate region, a body region and a solder paste region; The repairing the data missing region according to the positional relationship between the data missing region and the plurality of target regions, the angle value of each point in the data missing region and the effective value to obtain repaired point cloud data of the workpiece to be measured comprises: If the data missing region is located in the substrate region, determining the repaired point cloud data of each point according to a comparison relationship between the angle value of each point and a first preset value; If the data missing region is located in a middle region of the body region or a middle region of the solder paste region, determining the repaired point cloud data of each point according to the angle value of each point; If the data missing region is located in other regions of the body region or other regions of the solder paste region, determining the repaired point cloud data of each point according to the angle value of each point and the effective value.

4. The data repair method of claim 3, wherein, The determining the repaired point cloud data of each point according to a comparison relationship between the angle value of each point and a first preset value comprises: Setting the repaired point cloud data corresponding to a point in the data missing region, whose angle value is less than the first preset value, as a preset value; Setting the repaired point cloud data corresponding to a point in the data missing region, whose angle value is greater than or equal to the first preset value, as the mean value of the effective value thereof.

5. The data repair method of claim 3, wherein, The determining the repaired point cloud data of each point according to the angle value of each point and the effective value comprises: If the angle value is consistent with the angle value corresponding to the effective value thereof, setting the repaired point cloud data of the point as the mean value of the effective value thereof. If the angle value is inconsistent with the angle value corresponding to the effective value corresponding to the angle value, the point cloud data of the repaired point is determined as the effective value corresponding to the angle value.

6. The data repair method of claim 1, wherein, The plurality of target regions include a substrate region, a body region, and a solder paste region. The segmentation processing of the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm includes: Segmentation processing of the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm to generate a height area distribution map containing a plurality of regions; The region with the largest area and the lowest height value in the plurality of regions is determined as the substrate region. The region with color information being the preset color in the plurality of regions is determined as the solder paste region. The region with an area smaller than the substrate region, an area larger than the solder paste region, and the highest height value in the plurality of regions is determined as the body region. The region with a height value within a preset height range or an area smaller than a preset value in the plurality of regions is determined as the data missing region.

7. The data repair method of claim 6, wherein, The method further includes: According to the number of pixel points included in each region in the plurality of regions, the area corresponding to each region is determined. And / or, according to the average of the height values of the point cloud data included in each region in the plurality of regions, the height value corresponding to each region is determined.

8. The data repair method of claim 1, wherein, The method further includes: According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. According to the angle value of the pixel point, an angle compensation map is determined.

9. The data repair method of claim 8, wherein, According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. The angle calculation equation is: wherein, is an angle value of the pixel point, is a W value of the pixel point, "R" is an R value of the pixel point, "G" is a G value of the pixel point, and "B" is a B value of the pixel point.

10. The data repair method of claim 1, wherein, The determination of the effective value of each point in the converted point cloud data includes: Determination of the position information of each point in the data missing region; According to the position information of each point, the effective data in the eight neighborhood directions of each point is determined; the effective data is the point cloud data included in the target region; According to the effective data, the effective value of each point is determined.

Images