Data repair method

Abstract

The application discloses a data repairing method, comprising: segmenting an RGBW image according to point cloud data of a workpiece to be measured, the RGBW image and a preset segmentation algorithm, determining a plurality of target regions and a data missing region; fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; determining point cloud data located around each point in the data missing region in the converted point cloud data, and determining an effective value of each point; repairing the data missing region according to a positional relationship between the data missing region and the plurality of target regions, an angle value and the effective value of each point in the data missing region, and obtaining repaired point cloud data of the workpiece to be measured. Thus, corresponding repairing modes are collected for data missing regions located in different target regions, so that the repaired point cloud data of the workpiece to be measured can better restore the actual height of the workpiece to be measured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional automatic optical inspection equipment, and more particularly to a data repairing method. BACKGROUND

[0002] The workpiece to be measured is provided with electronic components (for example, resistors, capacitors, etc.), and the point cloud data of the workpiece to be measured is a set of height points of the components on the workpiece to be measured. The defects of the components, such as floating and virtual welding, can be detected through the point cloud data.

[0003] However, since the material of the components is usually highly reflective, light-absorbing, transparent or semi-transparent, the point cloud data of the workpiece to be measured may be lost. Using the point cloud data of the workpiece to be measured containing the loss condition cannot restore the true height thereof, which has a certain influence on detecting the defects of the components. SUMMARY

[0004] In view of the above problems, the present application provides a data repairing method to solve the above problems.

[0005] The data repairing method provided by the present application comprises the following steps: obtaining point cloud data and an RGBW image of a workpiece to be measured; performing segmentation processing on the RGBW image according to the point cloud data, the RGBW image and a preset segmentation algorithm to determine a plurality of target regions and a data loss region; fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; determining point cloud data located around each point in the data loss region in the converted point cloud data, and determining an effective value of each point; repairing the data loss region according to the positional relationship between the data loss region and the plurality of target regions, an angle value and the effective value of each point in the data loss region to obtain repaired point cloud data of the workpiece to be measured. Thus, by determining the data loss region contained in different target regions, a targeted repairing method is adopted to make the repaired point cloud data of the workpiece to be measured more complete, so as to better restore the true height of the workpiece to be measured, thereby reducing the influence on the detection of the workpiece to be measured and other related operations. BRIEF DESCRIPTION OF DRAWINGS

[0006] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by those skilled in the art without creative labor are within the scope of the present application.

[0007] Figure 1A structural schematic diagram of a three-dimensional automatic optical inspection device is shown.

[0008] Figure 2 A flowchart of a data repairing method is shown.

[0009] Figure 3 A structural schematic diagram of an RGBW image is shown.

[0010] Figure 4 A structural schematic diagram of a data repairing device is shown.

[0011] Figure 5 A structural schematic diagram of a three-dimensional automatic optical inspection device is shown.

[0012] Figure 6 A structural schematic diagram of a computer readable storage medium is shown. DETAILED DESCRIPTION

[0013] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application.

[0014] The workpiece to be measured is provided with electronic components (for example, resistors, capacitors, etc.), and the point cloud data of the workpiece to be measured is a group of height points of the components on the workpiece to be measured. The defects of the components, such as floating and virtual welding, can be detected through the point cloud data.

[0015] However, since the material of the components is usually highly reflective, light-absorbing, transparent or semi-transparent, the point cloud data of the workpiece to be measured may be lost. Using the point cloud data of the workpiece to be measured containing the loss cannot restore the true height thereof, which has a certain impact on detecting the defects of the components.

[0016] In order to improve the above problems, the present application provides a data repairing method, which comprises: acquiring point cloud data and an RGBW image of a workpiece to be measured; performing segmentation processing on the RGBW image according to the point cloud data, the RGBW image and a preset segmentation algorithm, to determine a plurality of target regions and a data loss region; fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; determining point cloud data located around each point in the data loss region in the converted point cloud data, and determining an effective value of each point; and repairing the data loss region according to a positional relationship between the data loss region and the plurality of target regions, an angle value and the effective value of each point in the data loss region, to obtain repaired point cloud data of the workpiece to be measured.

[0017] Therefore, by determining the data missing area contained in different target areas, a targeted repair method is adopted to make the point cloud data of the repaired workpiece more complete, so that the real height of the workpiece can be better restored, thereby reducing the influence on the detection of the workpiece and related operations.

[0018] The application environment of the data repair method provided in the embodiments of the application is introduced below.

[0019] Please refer to Figure 1 , Figure 1 The structure of a three-dimensional automatic optical inspection device is shown in the embodiments of the application, as shown in Figure 1 The three-dimensional automatic optical inspection device 100 includes a light source, a light machine 110, and an image acquisition module 120.

[0020] The light source is used to irradiate the workpiece located at the detection position of the three-dimensional automatic optical inspection device 100; the light machine 110 is used to scan the workpiece to collect the point cloud data of the workpiece; and the image acquisition module 120 is used to collect the RGBW image of the workpiece.

[0021] In some embodiments, the three-dimensional automatic optical inspection device 100 can be a 3D AOI.

[0022] In some embodiments, the light source can include an RGB light source and a W light source.

[0023] In some embodiments, the image acquisition module 120 can be a camera.

[0024] In the embodiments of the application, the three-dimensional automatic optical inspection device 100 can include a preset number of light machines 110. For example, the three-dimensional automatic optical inspection device 100 can include four light machines 110. For another example, the three-dimensional automatic optical inspection device 100 can include eight light machines 110. It can be understood that the specific number of light machines 110 is not limited in the application.

[0025] In the embodiments of the application, the three-dimensional automatic optical inspection device 100 can include four light machines 110, and the four light machines 110 are located at the upper side, the lower side, the left side, and the right side of the workpiece, respectively, to obtain more complete point cloud data of the workpiece.

[0026] By reasonably setting the light machine 110, the problem of incomplete point cloud data in the shadow area of the workpiece can be effectively solved. By scanning the workpiece through multiple light machines 110 located in different directions, the collected point cloud data is fused to obtain the point cloud data of the workpiece.

[0027] In some embodiments, the workpiece to be tested can be a PCB board, and the PCB board can further be provided with elements. The workpiece to be tested can be placed on a detection position of the three-dimensional automatic optical detection device 100, so as to collect point cloud data corresponding to the workpiece to be tested by the optical machine 110, and collect an RGBW image corresponding to the workpiece to be tested by the image acquisition module 120.

[0028] In some embodiments, the point cloud data of the workpiece to be tested can be a set of height values of the workpiece to be tested, i.e., height values of the elements provided on the workpiece to be tested. The point cloud data can be used to detect defects of the elements, such as floating, virtual welding, etc.

[0029] However, when the material of the elements on the workpiece to be tested is high-reflective, light-absorbing, transparent or semi-transparent, the point cloud data scanned by the optical machine 110 can be lost, so that the real height of the elements cannot be restored according to the point cloud data, and the related operations on the elements, such as detection of the elements, are affected. Therefore, it is necessary to repair the point cloud data of the workpiece to be tested obtained by scanning. Specifically:

[0030] Referring to Figure 2 , Figure 2 FIG. 7 shows a flowchart of a data repair method provided by an embodiment of the present application, which can be applied to the three-dimensional automatic optical detection device described above. As shown in Figure 2 , the method can include steps 210 to 220.

[0031] In step 210, point cloud data and an RGBW image of a workpiece to be tested are acquired.

[0032] The three-dimensional automatic optical detection device acquires point cloud data of the workpiece to be tested by a plurality of optical machines, and fuses the point cloud data acquired by the plurality of optical machines, so as to obtain the point cloud data of the workpiece to be tested.

[0033] The three-dimensional automatic optical detection device acquires an image of the workpiece to be tested by the image acquisition module, so as to obtain an RGBW image of the workpiece to be tested.

[0034] In subsequent steps, according to height values of elements included in the point cloud data of the workpiece to be tested and color information included in the RGBW image, the RGBW image is segmented to divide the RGBW image into a plurality of target regions and a plurality of data missing regions respectively included in the plurality of target regions, so that a specific repair method is adopted for the data missing regions located in different target regions, so that the point cloud data of the workpiece to be tested obtained after the repair is more complete, so as to better restore the real height of the workpiece to be tested, so as to reduce the influence on the detection of the workpiece to be tested and other related operations.

[0035] In step 220, the RGBW image is segmented according to the point cloud data, the RGBW image, and a preset segmentation algorithm to determine a plurality of target regions and data missing regions.

[0036] The preset segmentation algorithm can complete segmentation of the RGBW image according to height data included in the point cloud data and color information included in the RGBW image to determine a plurality of target regions on the RGBW image and data missing regions included in the plurality of target regions. In some embodiments, the preset segmentation algorithm can be a clustering segmentation algorithm. It can be understood that the application does not limit the specific algorithm of the preset segmentation algorithm.

[0037] In some embodiments, the plurality of target regions can include a substrate region, a body region, and a solder paste region on the PCB. That is, the data missing region can be located on the substrate region, the body region, or the solder paste region. By segmenting the point cloud data in the data missing region located on the substrate region, the body region, and the solder paste region, the point cloud data of the repaired workpiece to be tested is more complete, so that the real height of the workpiece to be tested can be better restored, thereby reducing detection and other related operations on the workpiece to be tested.

[0038] For example, referring to Figure 3 , Figure 3 FIG. 1 shows a structure of an RGBW image according to an embodiment of the application. As shown in FIG. 1, the RGBW image includes a substrate region A, a body region B, a solder paste region C, and a data missing region D. Figure 3

[0039] Specifically, in some embodiments, the step of “segmenting the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm to determine a plurality of target regions and data missing regions” can include the following steps:

[0040] (1) Segmenting the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm to generate a height area distribution map containing a plurality of regions.

[0041] (2) Determining a region with the largest area and the lowest height value in the plurality of regions as the substrate region.

[0042] (3) Determining a region with color information being a preset color in the plurality of regions as the solder paste region.

[0043] (4) Determining a region with an area smaller than the substrate region, an area larger than the solder paste region, and a highest height value in the plurality of regions as the body region.

[0044] (5) Determining a region with a height value within a preset height range or an area smaller than a preset value in the plurality of regions as the data missing region.​

[0045] In some embodiments, the three-dimensional automatic optical inspection device determines the area corresponding to each of the plurality of regions according to the number of pixels included in each of the plurality of regions. That is, the areas of the substrate region, the body region, the solder paste region, and the data missing region are determined respectively.

[0046] In some embodiments, the three-dimensional automatic optical inspection device determines the height value corresponding to each of the plurality of regions according to the mean value of the height values of the point cloud data included in each of the plurality of regions. That is, the height values of the substrate region, the body region, the solder paste region, and the data missing region are determined respectively.

[0047] After determining the area and the height value corresponding to each of the plurality of regions, the three-dimensional automatic optical inspection device determines the specific type of each of the plurality of regions according to the area and the height value corresponding to each of the plurality of regions. That is, the substrate region, the body region, the solder paste region, or the data missing region is determined.

[0048] In some embodiments, the preset color can be blue. The three-dimensional automatic optical inspection device determines the region with the color information of blue on the RGBW image as the solder paste region.

[0049] In some embodiments, the preset height range can be the interval (-7000, 30000). It can be understood that the specific interval of the preset height range is not limited in the present application. The three-dimensional automatic optical inspection device determines the region surrounded by the point cloud data with the height data located in the interval (-7000, 30000) as the data missing region according to the point cloud data of the workpiece to be measured.

[0050] In some embodiments, the preset value can be 100. It can be understood that the specific value of the preset value is not limited in the present application. After determining the areas of the plurality of regions, the three-dimensional automatic optical inspection device determines the region with the area less than 100 as the data missing region.

[0051] That is, the three-dimensional automatic optical inspection device determines the region surrounded by the point cloud data with the height data located in the interval (-7000, 30000) as the data missing region, or determines the region with the area less than 100 as the data missing region in the plurality of regions.

[0052] The three-dimensional automatic optical inspection device determines the region with the largest area and the lowest height value of the point cloud data as the substrate region in the plurality of regions, and determines the region with the second largest area and the highest height value as the body region.

[0053] The three-dimensional automatic optical inspection equipment determines the substrate area, the body area, the solder paste area and the data missing area in the height area distribution map through the above separation method. It can be understood that, by determining the data missing area in the height area distribution map and directly repairing all the data missing areas uniformly, the repaired point cloud data of the workpiece to be measured can be obtained. According to the present application, the substrate area, the body area and the solder paste area are determined in the height area distribution map, and then it is determined which area the data missing area is located in the substrate area, the body area and the solder paste area. Then, according to the area where the data missing area is located, a targeted repair method is collected to make the repaired point cloud data closer to the most real situation of the workpiece to be measured, improve the accuracy of the repaired point cloud data, and better restore the workpiece to be measured. Specifically:

[0054] In step 230, a zero plane is fitted according to the point cloud data and the least square method, and the point cloud data is converted to the zero plane to obtain converted point cloud data.

[0055] Before the three-dimensional automatic optical inspection equipment adopts a targeted repair method based on which area the data missing area is located in the substrate area, the body area and the solder paste area, the point cloud data corresponding to the workpiece to be measured needs to be unified into the same coordinate system.

[0056] The three-dimensional automatic optical inspection equipment substitutes the point cloud data of the workpiece to be measured into the least square method to find a best fitting plane, i.e. a zero plane, in the point cloud data of the workpiece to be measured. After determining the zero plane corresponding to the point cloud data, the point cloud data is converted to the zero plane to obtain converted point cloud data, thereby realizing the unification of the point cloud data of the workpiece to be measured into the same coordinate system.

[0057] In order to reduce the influence of outliers in the collected point cloud data of the workpiece to be measured, and to reduce the error of the zero plane fitted by the point cloud data of the workpiece to be measured. Based on the above, in some embodiments, the three-dimensional automatic optical inspection equipment sorts the point cloud data of the workpiece to be measured according to the corresponding height value from high to low to obtain a sorted point cloud data set; removes the first 10% and the last 10% of the point cloud data in the sorted point cloud data set to obtain a final point cloud data set. Then, according to the final point cloud data set and the least square method, a zero plane is fitted, and the final point cloud data set is converted to the zero plane to complete the unification of the coordinate system of the final point cloud data.

[0058] Based on the point cloud data after unifying the coordinate system, the three-dimensional automatic optical inspection equipment determines the effective value corresponding to each pixel point in the data missing area, so as to correct according to the effective value corresponding to the pixel point in the subsequent steps. Specifically:

[0059] In step 240, in the converted point cloud data, the point cloud data located around each point in the data missing area is determined, and the effective value of each point is determined.

[0060] The three-dimensional automatic optical detection device finds the closest effective data to each point in the data missing area based on the surroundings of each point in the data missing area, that is, determines the point in the substrate area, the body area or the solder paste area closest to each point in the data missing area, and records it to determine the effective value of each point.

[0061] Specifically, in some embodiments, the step of "in the converted point cloud data, determining the point cloud data located around each point in the data missing area, and determining the effective value of each point" can include the following steps:

[0062] (1) Determine the position information of each point in the data missing area.

[0063] (2) According to the position information of each point, determine the effective data in the eight-neighborhood direction of each point; the effective data is the point cloud data included in the target area.

[0064] (3) According to the effective data, determine the effective value of each point.

[0065] After the three-dimensional automatic optical detection device obtains the position information of each pixel point in the data missing area, it searches in the eight-neighborhood direction based on the position information of each pixel point, finds the effective data (i.e. point cloud data in the substrate area, body area or solder paste area) of each pixel point in each direction, records the coordinates and height values of the effective positions, and determines the effective value of each point. Further:

[0066] In step 250, according to the positional relationship between the data missing area and the plurality of target areas, the angle value and the effective value of each point in the data missing area, the data missing area is repaired to obtain the repaired point cloud data of the workpiece to be measured.

[0067] The positional relationship between the data missing area and the plurality of target areas, that is, the data missing area is located in the substrate area, the body area or the solder paste area, so as to collect the targeted repair method according to which area the data missing area is located in.

[0068] The angle value of each point in the data missing area can be determined by the angle compensation map corresponding to the RGBW image. Specifically, in some embodiments, the data repair method can include the following steps:

[0069] (1) Calculate the angle value of the pixel point according to the R value, G value, B value and W value of the pixel point in the RGBW image to determine the angle value of the pixel point.

[0070] (2) determining an angle compensation map according to the angle value of the pixel point.

[0071] The three-dimensional automatic optical inspection equipment determines the angle value corresponding to each pixel point in the RGBW image according to the R value, G value, B value or W value corresponding to each pixel point in the RGBW image, so as to determine the angle compensation map of the workpiece to be measured.

[0072] Specifically, in some embodiments, the step of "calculating the angle value of the pixel point according to the R value, G value, B value and W value corresponding to the pixel point in the RGBW image to determine the angle value of the pixel point" can include the step of: determining the angle value of the pixel point according to the R value, G value, B value, W value of the pixel point and the angle calculation equation.

[0073] The angle calculation equation can be expressed as:

[0074]

[0075]

[0076] The angle value of the pixel point is represented by "a", the W value of the pixel point is represented by "w", the R value of the pixel point is represented by "r", the G value of the pixel point is represented by "g", and the B value of the pixel point is represented by "b".

[0077] The three-dimensional automatic optical inspection equipment determines the angle value corresponding to each pixel point in the RGBW image according to the R value, G value, B value or W value corresponding to each pixel point in the RGBW image, so as to determine the angle compensation map of the workpiece to be measured.

[0078] After determining the angle value corresponding to each pixel point in the RGBW image, the three-dimensional automatic optical inspection equipment combines the seed filling algorithm to repair each data missing area to obtain the point cloud data corresponding to each data missing area. Specifically:

[0079] In some embodiments, the step of "repairing the data missing area according to the positional relationship between the data missing area and the plurality of target areas, the angle value and the effective value of each point in the data missing area, to obtain the repaired point cloud data of the workpiece to be measured" can include the following steps:

[0080] (1) If the data missing area is located in the substrate area, the repaired point cloud data of each point is determined according to the comparison relationship between the angle value of each point and the first preset value.

[0081] ​​​​​(2) If the data missing region is located in the middle region of the body region or the middle region of the solder paste region, the point cloud data of each point after repair is determined according to the angle value of each point.

[0082] (3) If the data missing region is located in other regions of the body region or other regions of the solder paste region, the point cloud data of each point after repair is determined according to the angle value and the effective value of each point.

[0083] In some embodiments, the step of "determining the point cloud data of each point after repair according to the comparison relationship between the angle value of each point and the first preset value" can include the following steps:

[0084] (1) The point cloud data corresponding to the points in the data missing region with an angle value less than the first preset value is set to a preset value.

[0085] (2) The point cloud data corresponding to the points in the data missing region with an angle value greater than or equal to the first preset value is set to the mean value of its effective value.

[0086] In some embodiments, the first preset value can be 0. It can be understood that the specific value of the first preset value is not limited in the present application.

[0087] In some embodiments, the preset value can be 0. It can be understood that the specific value of the preset value is not limited in the present application.

[0088] If the angle value corresponding to the pixel points in the substrate region is less than 0, indicating that these data points are abnormal values or noises, the three-dimensional automatic optical detection equipment sets the point cloud data corresponding to the pixel points to 0. If the angle value corresponding to the pixel points in the substrate region is greater than or equal to 0, the three-dimensional automatic optical detection equipment sets the point cloud data corresponding to the pixel points to the mean value of the effective value corresponding to the pixel points. This method can help to fill in the missing values or correct possible errors by filling the effective values around the data points, thereby ensuring the continuity and integrity of the data.

[0089] Since the substrate region has high surface flatness or low defects, the three-dimensional automatic optical detection equipment can make the data points in the substrate region smoothly transition by using the above repair method, and can effectively maintain the structural characteristics of the substrate region to complete the repair of the point cloud data corresponding to the substrate region.

[0090] The middle region of the body region or the middle region of the solder paste region can be pre-set, for example, in the body region and the solder paste region, the region overlapping with the RGBW image designated region is determined as the middle region of the body region and the middle region of the solder paste region.

[0091] If the data missing region is located in the middle region of the body region or the middle region of the solder paste region, the point cloud data corresponding to each point in the data missing region is estimated according to the angle value corresponding to each point.

[0092] Specifically, the middle region of the solder paste region is defined as a starting point, and the middle region of the body region is defined as an ending point, and the middle region of the solder paste region is filled with the angle value corresponding to each point. It can be understood that the starting point and the ending point are not limited in the present application, for example, starting from one end of the middle region of the solder paste region and filling to the other end of the middle region of the solder paste region.

[0093] If the data missing region is located in other regions of the body region or other regions of the solder paste region, i.e. not located in the middle region of the body region or the middle region of the solder paste region, the point cloud data of each point after repair is determined according to whether the angle value of each point is consistent with the angle value corresponding to the valid value.

[0094] Further, in some embodiments, the step "if the data missing region is located in other regions of the body region or other regions of the solder paste region, then the point cloud data of each point after repair is determined according to the angle value of each point and the valid value" can include the following steps:

[0095] (1) If the angle value is consistent with the angle value corresponding to the valid value corresponding to the angle value, the point cloud data of the point after repair is determined as the mean value of the valid value corresponding to the angle value.

[0096] (2) If the angle value is not consistent with the angle value corresponding to the valid value corresponding to the angle value, the point cloud data of the point after repair is determined as the valid value corresponding to the angle value.

[0097] The three-dimensional automatic optical inspection equipment determines whether the angle value corresponding to the valid value corresponding to each data point in the data missing region located in other regions of the body region or other regions of the solder paste region is consistent with the angle value corresponding to the angle compensation graph, if it is consistent, it means that the angle value corresponding to the valid value corresponding to the data point and the angle value corresponding to the angle compensation graph has similar directionality or angle characteristics, and the data is filled by the mean value of the valid value corresponding to the data point, which can make the filled data more smooth and natural, and keep the original angle characteristics.

[0098] If the consistency is not met, it means that the angle value corresponding to the effective value of the data point and the angle value corresponding to the data point in the angle compensation graph have different directionality or angle characteristics, which may indicate that the region formed by the data point and its attached data points has a unique geometric structure, surface change or other physical phenomenon. At this time, if the mean value of the effective value corresponding to the data point is used to fill it, errors may be introduced because the local variation information is ignored. Therefore, in this case, the effective value corresponding to the data point is used to fill it to better reflect the actual geometric shape or surface attribute, rather than simply "smoothing" the difference.

[0099] It is worth noting that in the embodiments of the present application, the three-dimensional automatic optical detection equipment preferentially fills the data missing area with the smallest height difference between the effective values of the contained data points. By preferentially repairing the data missing area with the smallest height difference between the effective values, it is marked as valid data, which can expand the range of valid data, so that other data points can be filled faster to correspond to the point cloud data.

[0100] After the three-dimensional automatic optical detection equipment completes the repair of the point cloud data of the to-be-detected data in the above-mentioned manner, in order to further detect whether the repaired point cloud data still has a data missing condition, and fill the missing data points in time. In some embodiments, the data repair method can further include the following steps:

[0101] (1) updating the point cloud data to the repaired point cloud data;

[0102] (2) according to the updated point cloud data, the RGBW image and the preset segmentation algorithm, the RGBW image is segmented and processed to determine whether there is a data missing area.

[0103] (2) if there is a data missing area, step (2) is executed according to the point cloud data and the least square method, fitting a zero plane, and converting the point cloud data to the zero plane to obtain the converted point cloud data.

[0104] The three-dimensional automatic optical detection equipment performs a second round of segmentation processing on the RGBW image according to the repaired point cloud data of the to-be-detected workpiece, the RGBW image and the preset segmentation algorithm, to determine whether there is a point cloud data missing condition in the current to-be-detected workpiece. If there is a point cloud data missing condition, the missing point cloud data is repaired according to the above-mentioned point cloud data repair method until there is no point cloud data missing condition.

[0105] That is, in the present application, the point cloud data of the workpiece under test can be repaired by one round. The point cloud data of the workpiece under test can also be repaired by at least two rounds. The point cloud data of the workpiece under test after final repair is more complete, so that the real height of the workpiece under test can be better restored, thereby reducing the accuracy of detection and other related operations of the workpiece under test.

[0106] Please refer to Figure 4 , Figure 4 A structure diagram of a data repair device provided by an embodiment of the present application is shown, which is applied to the three-dimensional automatic optical detection equipment described above. The data repair device 300 includes an acquisition module 310, a first execution module 320, a second execution module 330, a determination module 340 and a third execution module 350, specifically:

[0107] The acquisition module 310 is configured to acquire point cloud data and an RGBW image of a workpiece under test.

[0108] The first execution module 320 is configured to perform segmentation processing on the RGBW image according to the point cloud data, the RGBW image and a preset segmentation algorithm, to determine a plurality of target regions and a data missing region.

[0109] The second execution module 330 is configured to fit a zero plane according to the point cloud data and a least square method, and convert the point cloud data to the zero plane to obtain converted point cloud data.

[0110] The determination module 340 is configured to determine point cloud data located around each point in the data missing region in the converted point cloud data, and determine an effective value of each point.

[0111] The third execution module 350 is configured to repair the data missing region according to a positional relationship between the data missing region and the plurality of target regions, an angle value and the effective value of each point in the data missing region, to obtain repaired point cloud data of the workpiece under test.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and module can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0113] In several embodiments provided in the present application, the coupling or direct coupling or communication connection between the modules displayed or discussed can be indirect coupling or communication connection between the interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0114] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can be physically present alone, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0115] Please refer to Figure 5 , Figure 5 A structure diagram of a three-dimensional automatic optical inspection device provided by an embodiment of the present application is shown. The three-dimensional automatic optical inspection device 400 in the present application can include one or more of the following components: a processor 410, a memory 420, and one or more application programs, wherein the one or more application programs can be stored in the memory 420 and configured to be executed by the one or more processors 410, and the one or more programs are configured to execute the data repair method as described in the foregoing method embodiments.

[0116] The processor 410 can include one or more processing cores. The processor 410 connects various parts in the entire three-dimensional automatic optical inspection device 400 through various interfaces and lines, executes various functions of the three-dimensional automatic optical inspection device 400 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 420, and calling data stored in the memory 420. Optionally, the processor 410 can be realized in the form of at least one of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 410 can integrate a combination of one or several of central processing units (CPUs), graphics processing units (GPUs), and modems. Among them, the CPU is mainly used to process operating systems, user interfaces, and application programs; the GPU is used to render and draw display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 410, but realized by a separate communication chip.

[0117] The memory 420 can include a Random Access Memory (RAM) and can also include a Read-Only Memory (ROM). The memory 420 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 420 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing each of the methods described below, and the like. The data storage area can also store data created by the three-dimensional automatic optical inspection apparatus 400 in use.

[0118] Referring to Figure 6 , Figure 6 A structure diagram of a computer-readable storage medium according to an embodiment of the present application is shown. The computer-readable storage medium 500 stores program codes, which can be invoked by a processor to execute the data repair method described in the above method embodiments.

[0119] The computer-readable storage medium 500 can be an electronic storage such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has a storage space for program codes 510 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program devices. The program codes 510 can be compressed in an appropriate form, for example.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data repair method characterized by, The method is applied to a three-dimensional automatic optical inspection device, and comprises the following steps: Obtaining point cloud data and an RGBW image of a workpiece to be measured; Segmenting the RGBW image according to the point cloud data, the RGBW image and a preset segmentation algorithm to determine a plurality of target regions and a data missing region; the target regions include a substrate region, a body region and a solder paste region; Fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data; In the converted point cloud data, determining point cloud data located around each point in the data missing region and determining effective values of the each point; Repairing the data missing region according to a positional relationship between the data missing region and the plurality of target regions, an angle value of each point in the data missing region and the effective values to obtain repaired point cloud data of the workpiece to be measured; The repairing the data missing region according to the positional relationship between the data missing region and the plurality of target regions, the angle value of each point in the data missing region and the effective values to obtain the repaired point cloud data of the workpiece to be measured comprises: If the data missing region is located in the substrate region, determining repaired point cloud data of the each point according to a comparison relationship between the angle value of the each point and a first preset value; If the data missing region is located in a middle region of the body region or a middle region of the solder paste region, determining the repaired point cloud data of the each point according to the angle value of the each point; If the data missing region is located in other regions of the body region or other regions of the solder paste region, determining the repaired point cloud data of the each point according to the angle value of the each point and the effective values.

2. The data repair method of claim 1, wherein, The method further comprises: Updating the point cloud data to the repaired point cloud data; Segmenting the RGBW image according to the updated point cloud data, the RGBW image and the preset segmentation algorithm to determine whether the data missing region exists; If the data missing region exists, performing the step of fitting a zero plane according to the point cloud data and a least square method, and converting the point cloud data to the zero plane to obtain converted point cloud data.

3. The data repair method of claim 1, wherein, The determining the repaired point cloud data of the each point according to the comparison relationship between the angle value of the each point and the first preset value comprises: Setting repaired point cloud data corresponding to a point in the data missing region, whose angle value is less than the first preset value, to a preset value; Setting repaired point cloud data corresponding to a point in the data missing region, whose angle value is greater than or equal to the first preset value, to a mean value of effective values of the point.

4. The data repair method of claim 1, wherein, The determining the repaired point cloud data of the each point according to the angle value of the each point and the effective values comprises: If the angle value is consistent with an angle value corresponding to an effective value corresponding to the point, setting repaired point cloud data of the point to a mean value of the effective value. If the angle value is inconsistent with the angle value corresponding to the effective value corresponding to the angle value, the point cloud data of the point after repair is determined as the effective value corresponding to the angle value.

5. The data repair method of claim 1, wherein, The plurality of target regions include a substrate region, a body region, and a solder paste region. The segmentation processing of the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm includes: Segmentation processing of the RGBW image according to the point cloud data, the RGBW image, and the preset segmentation algorithm to generate a height area distribution map containing a plurality of regions; The region with the largest area and the lowest height value in the plurality of regions is determined as the substrate region. The region with the color information being the preset color in the plurality of regions is determined as the solder paste region. The region with the area being smaller than the substrate region, the area being larger than the solder paste region, and the height value being the highest in the plurality of regions is determined as the body region. The region with the height value being within a preset height range or the area being smaller than a preset value in the plurality of regions is determined as the data missing region.

6. The data repair method of claim 5, wherein, The method further includes: According to the number of pixel points included in each region in the plurality of regions, the area corresponding to each region is determined. And / or, according to the average of the height values of the point cloud data included in each region in the plurality of regions, the height value corresponding to each region is determined.

7. The data repair method of claim 1, wherein, The method further includes: According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. According to the angle value of the pixel point, an angle compensation map is determined.

8. The data repair method of claim 7, wherein, According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. According to the R value, G value, B value, and W value of the pixel point in the RGBW image, the angle value of the pixel point is calculated to determine the angle value of the pixel point. The angle calculation equation is: wherein, is an angle value of the pixel point, is a W value of the pixel point, "R" is an R value of the pixel point, "G" is a G value of the pixel point, and "B" is a B value of the pixel point.

9. The data repair method of claim 1, wherein, The determination of the effective value of each point in the converted point cloud data includes: Determination of the position information of each point in the data missing region; According to the position information of each point, the effective data located in the eight neighborhood directions of each point is determined; the effective data is the point cloud data included in the target region; According to the effective data, the effective value of each point is determined.

Images