Defect detection method and device of target part, computer equipment and storage medium

Abstract

The invention relates to a target part defect detection method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an initial depth image of a target piece, and determining image units divided by the initial depth image; determining a reference fitting area extended by the image unit according to the extension range, selecting a plurality of points in the reference fitting area along the target direction, and performing fitting according to the plurality of points in the reference fitting area to obtain a reference model; mapping the image unit according to the reference model to obtain a mapped image unit; and performing defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target piece. By adopting the method and the device, the efficiency and the accuracy of defect detection of the free-form surface can be improved.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a method, device, computer equipment and storage medium for defect detection of a target part. Background Art

[0002] With the continuous development of industrial production, there are more and more free-form surfaces in industrial products, and defect detection methods are needed to detect defects in free-form surfaces.

[0003] In traditional technology, the collected industrial product images are compared with defect-free industrial product images, and the corresponding defects are obtained based on the phase difference. However, in practical applications, there are many types of free-form surfaces and defects, and adaptive adjustments may be required based on factors such as surface shape and lighting conditions, resulting in low processing efficiency. Summary of the invention

[0004] Based on this, it is necessary to provide a target part defect detection method, device, computer equipment, storage medium and computer program product to address the above technical problems, which can improve the efficiency and accuracy of free-form surface defect detection.

[0005] In a first aspect, the present application provides a method for defect detection of a target part, comprising:

[0006] Acquire an initial depth image of the target part, and determine image units into which the initial depth image is divided;

[0007] Determine a reference fitting region extending from the image unit according to the extension range, select multiple points in the reference fitting region along the target direction, and perform fitting according to the multiple points in the reference fitting region to obtain a reference model;

[0008] According to the benchmark model, the image unit is mapped to obtain a mapped image unit;

[0009] According to the mapped image units, defect detection is performed on the initial depth image to obtain the defect detection result of the target part.

[0010] In a second aspect, the present application provides a defect detection device for a target part, comprising:

[0011] An acquisition module, used to acquire an initial depth image of the target part and determine image units into which the initial depth image is divided;

[0012] A fitting module, used to determine a reference fitting area extending from the image unit according to the extension range, select a plurality of points in the reference fitting area along the target direction, and perform fitting according to the plurality of points in the reference fitting area to obtain a reference model;

[0013] A mapping module, used for mapping the image unit according to the reference model to obtain the mapped image unit;

[0014] The detection module is used to perform defect detection on the initial depth image according to the mapped image unit to obtain the defect detection result of the target part.

[0015] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0016] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0017] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0018] The above-mentioned defect detection method, device, computer equipment, storage medium and computer program product of the target part obtain the initial depth image of the target part, and determine the image unit divided by the initial depth image, and control the pixel granularity of the initial depth image through the image unit to control the accuracy and efficiency of subsequent fitting and mapping; then determine the reference fitting area extended by the image unit according to the extension range, efficiently determine multiple points in the reference fitting area along the target direction, and fit according to the multiple points in the reference fitting area to obtain a reference model; apply the image data of the image unit neighborhood to the model fitting process, and use the extended reference fitting area to calculate the surface, and can regulate and ensure that the efficiency and accuracy are balanced through the extension range. Then, according to the reference model, the image unit is mapped to obtain the mapped image unit; because the reference model is obtained by fitting the reference fitting area of ​​each image unit, the image unit can be converted into a theoretical mapped image unit, which can reflect the optimal state of the corresponding industrial product in the image unit, and then according to the mapped image unit, the initial depth image is defect detected to accurately judge the defects of the target part. In this way, the efficiency and accuracy of defect detection of free-form surfaces can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 An application environment diagram of a target part defect detection method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a process flow of a target component defect detection method provided in an embodiment of the present application;

[0021] Figure 3A schematic diagram of an initial depth image and an image unit provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of an image unit and a reference fitting area provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a process of fitting a benchmark model provided in an embodiment of the present application;

[0024] Figure 6 A structural block diagram of a defect detection device for a target part provided in an embodiment of the present application;

[0025] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0026] Figure 8 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application.

[0027] Reference numerals

[0028] 301 - grid; 302 - detection area; 303 - background area; 401 - benchmark fitting area. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The target part defect detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud computing services.

[0031] like Figure 2 As shown, the embodiment of the present application provides a defect detection method for a target part, and the method is applied to Figure 1The terminal 102 in the example is or the server 104 is used as an example. It can be understood that the computer device can include at least one of the terminal and the server. The method includes the following steps:

[0032] Step 202: Acquire an initial depth image of the target part, and determine the image units into which the initial depth image is divided.

[0033] The target part is the object to be inspected. The target part is an object with at least one surface. For example, the target part can be any object with any curved surface; the target part includes but is not limited to a lens with a lens, a glass product with a lens, an irregular part or other product.

[0034] The initial depth image represents the depth information of the target part relative to the shooting device. The initial depth image can convert the surface of the target part into multi-dimensional data with depth information to lay the foundation for subsequent data processing.

[0035] Specifically, when the initial depth image is in a two-dimensional form, the initial depth image includes a plurality of pixels, each pixel corresponds to at least one point on the target part, and the pixel value of each pixel represents the relative distance between at least one point on the target part and the shooting device. When the initial depth image is in a three-dimensional form, the initial depth image is point cloud data, each point in the point cloud data corresponds to at least one point on the target part, and the relative distance between at least one point on the target part and the shooting device can be represented by the value of each point in the point cloud data on a certain coordinate axis. For example, the point cloud data includes coordinate values ​​of coordinate axes x, y, and z, and the value of each point in the point cloud data on the z coordinate axis represents the relative distance between at least one point on the target part and the shooting device.

[0036] An image unit is a pixel set obtained by dividing pixels in the initial depth image according to pixel positions. An image unit includes multiple points in the initial depth image to form a corresponding pixel set.

[0037] Specifically, the pixels in each image unit are divided according to the pixel position; illustratively, the specification of each image unit can reflect the required accuracy and processing speed of the pixels in this area, and the specification of each image unit can be adjusted according to the accuracy requirements and processing speed requirements to ensure flexible adjustment of accuracy and speed. illustratively, the image unit can be each pixel set obtained by rasterizing according to a preset size.

[0038] For example, Figure 3As shown, in order to reduce the amount of data processing, it is only necessary to rasterize the detection area in the initial depth map to obtain a grid-shaped image unit (cell), and each small square is each divided grid 301; among them, the gray area is the detection area 302, that is, the area to be calculated, and the white area is the background area 303.

[0039] Specifically, since the surface trends of the neighborhoods of points within a certain range are similar, for each point in an image unit, it is only necessary to use at least one point in the unit as a unit reference point and perform a fit through the unit reference point and its neighborhood points to obtain a common reference model for each point in the image unit, thereby reducing the time consumption of the algorithm. The unit reference point is at least one pixel point in the image unit, which is used to represent the depth information of the image unit relative to the camera.

[0040] In some embodiments, obtaining an initial depth image of a target object includes:

[0041] Scan the target part with a 3D camera or a 3D scanner to obtain an initial depth image of the target part; or

[0042] From the terminal or server, obtain the initial depth image of the target part.

[0043] In some embodiments, determining the image units for the initial depth image segmentation includes:

[0044] According to a preset number of pixels, pixels in the initial depth image are evenly divided to obtain image units; or,

[0045] The pixels in the initial depth image are divided according to a preset length range where the positions of the pixels in the initial depth image are located to obtain image units.

[0046] Step 204 , determining a reference fitting region extending from the image unit according to the extension range, selecting a plurality of points in the reference fitting region along the target direction, and performing fitting according to the plurality of points in the reference fitting region to obtain a reference model.

[0047] The extension range is a pixel range related to the image unit. The extension range contains pixels of the image unit and the image unit at adjacent positions to form more extended areas for fitting. Specifically, the extension range can be a preset length or preset interval in different directions.

[0048] The reference fitting area is a set of pixels in the initial depth image that contains image units and adjacent positions of image units; the range of each image unit is less than or equal to the range of its own reference fitting area. There is a one-to-one correspondence between image units and reference fitting areas, and the pixels of the reference fitting area may exist in the image units. Each image unit has its corresponding reference fitting area to fit each image unit separately. Exemplarily, the reference fitting area may be a rectangular area, a circular area, or an area in the same row or column as the image unit. Image unit 301 and reference fitting area 401 are as follows: Figure 4 shown.

[0049] The reference model is a relational expression used to map image units. The reference model includes mapping coefficients, which are used to map pixel values ​​in the image units to estimate theoretical pixel values ​​according to the pixel distribution law of the reference fitting area. Exemplarily, the reference model can be obtained by surface fitting or curve fitting.

[0050] In some embodiments, determining the reference fitting area extending from the image unit according to the extension range includes:

[0051] In the initial depth image, taking the position of the image unit as the center, a position whose distance from the image unit is an extension length is selected as a boundary; based on the boundary, a reference fitting area where the image unit is located is determined.

[0052] In some embodiments, determining the reference fitting area extending from the image unit according to the extension range includes:

[0053] In the initial depth image, taking the position of the unit reference point as the center, select the position whose distance from the image unit is the extension length in different directions;

[0054] At this location, a reference fitting region boundary parallel to a boundary of the image unit is determined;

[0055] The reference fitting region where the image unit is located is determined according to the region enclosed by the boundary of the reference fitting region.

[0056] In some embodiments, selecting a plurality of points in a reference fitting region along a target direction, and performing fitting according to the plurality of points in the reference fitting region to obtain a reference model includes:

[0057] Selecting a plurality of support points along a target direction according to the depth image data in the reference fitting region;

[0058] According to the least square method, multiple support points are fitted to obtain the benchmark model.

[0059] In some embodiments, selecting a plurality of points in a reference fitting region along a target direction, and performing fitting according to the plurality of points in the reference fitting region to obtain a reference model includes:

[0060] Selecting multiple points in the reference fitting area along the target direction according to the scene;

[0061] The benchmark model is obtained by fitting multiple points in the benchmark fitting area.

[0062] In some embodiments, the method in the PCL library is compared with the present embodiment; the method in the PCL library includes:

[0063] Determine the neighboring points of each point, fit each point with its respective neighboring points along the target direction to obtain a fitting model for each point, map the point according to the fitting model of each point, and obtain a mapped image unit.

[0064] In this embodiment, a reference fitting area extending from each image unit is determined; fitting is performed based on multiple points along the target direction within the reference fitting area to obtain a reference model obtained by fitting each image unit; and pixel points of each image unit are mapped based on the reference model of each image unit to obtain a mapped image unit.

[0065] Assuming that the specification of the image unit is 5*5, including 25 units of pixel points, the method in this paper only fits the surface equation of the area once, while the method in the PCL library needs to fit 25 times to obtain a fitting model of 25 points. In this embodiment, only one fitting is required based on multiple points in the benchmark fitting area to obtain the benchmark model of the image unit.

[0066] In some embodiments, each image unit selects a point as a unit reference point, and the conventional moving least squares (MLS) is compared with the present embodiment; in the conventional moving least squares method, for each point, a point within a certain distance neighborhood is selected for reference surface fitting. When the size of the image unit is 8×8, when processing an area of ​​the size of an image unit, the time used for model fitting is approximately reduced to: 1-1 / (8*8)≈0.9843, which is reduced by about 98%, thereby ensuring the effect of efficiency optimization.

[0067] Step 206: Map the image unit according to the reference model to obtain a mapped image unit.

[0068] The mapped image unit is the theoretical image unit of the area where the image unit is located. Since the benchmark model is determined based on the image unit and the benchmark fitting area, each mapped image unit is based on the depth data estimated in the local area. The mapped image unit contains multiple points, each of which corresponds one-to-one to a point in the image unit, and is used to represent the theoretical pixel value of each point in the image unit, that is, the depth value.

[0069] In some embodiments, mapping the image unit according to the reference model to obtain the mapped image unit includes:

[0070] In the case where the image unit belongs to a two-dimensional image, the pixel value of the image unit at each position is mapped according to the weight of the reference model at each position to obtain a mapped image unit.

[0071] In some embodiments, mapping the image unit according to the reference model to obtain the mapped image unit includes:

[0072] In the case where the image unit belongs to a point cloud, the point of the image unit at each position is adjusted according to the weight of the reference model at each position to obtain a mapped image unit.

[0073] Step 208 , performing defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part.

[0074] In some embodiments, defect detection is performed on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part, including:

[0075] Compare the mapped image unit with the initial depth image to obtain the image difference;

[0076] According to whether the image difference meets the preset defect type, if the image difference meets the preset defect type, it is determined that the initial depth image has defects and the corresponding defect type; otherwise, it is determined that the initial depth image does not have defects.

[0077] In some embodiments, defect detection is performed on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part, including:

[0078] Performing differential processing on the mapped image unit and the initial depth image to obtain a distance map between the mapped image unit and the initial depth image;

[0079] According to the size of the distance, the concave and convex defects whose distance exceeds the index can be extracted.

[0080] In the above-mentioned defect detection method of the target part, the initial depth image of the target part is obtained, and the image unit divided by the initial depth image is determined. The pixel granularity of the initial depth image is controlled by the image unit to control the accuracy and efficiency of the subsequent fitting and mapping; then the reference fitting area extended by the image unit is determined according to the extension range, and multiple points in the reference fitting area are efficiently determined along the target direction, and fitting is performed according to the multiple points in the reference fitting area to obtain a reference model; the image data of the image unit neighborhood is applied to the model fitting process, and the surface is calculated by the extended reference fitting area, and the efficiency and accuracy can be regulated and guaranteed to be balanced through the extension range. Then, according to the reference model, the image unit is mapped to obtain the mapped image unit; since the reference model is obtained by fitting the reference fitting area of ​​each image unit, the image unit can be converted into a theoretical mapped image unit, which can reflect the optimal state of the corresponding industrial product in the image unit, and then the initial depth image is detected according to the mapped image unit to accurately judge the defects of the target part. In this way, the efficiency and accuracy of defect detection of free-form surfaces can be improved.

[0081] In some embodiments, Figure 5 As shown, fitting is performed based on multiple points in the benchmark fitting area to obtain a benchmark model, including:

[0082] Step 502 : Select multiple points in the reference fitting area along the target direction according to a preset step length to obtain multiple support points.

[0083] The preset step size is an interval value set for the reference fitting area, and the size of the interval value is related to the accuracy of the reference model. Specifically, the preset step size can be an interval set for the number of points in the reference fitting area, and multiple points in the reference fitting area are selected with the preset number of points as the interval; the preset step size can be an interval set for the length of the reference fitting area, and multiple points in the reference fitting area are selected with the preset length as the interval.

[0084] The support points are points selected from the reference fitting area for fitting. Specifically, when using the moving least squares method, each reference fitting area is first determined, and multiple support points in each reference fitting area are selected. Specifically, the support points can be points used for curve fitting or points used for surface fitting.

[0085] In some embodiments, when the preset step size is 0, all points are taken, and the preset step size is 2, that is, a point is taken every two pixels in the row and column direction. The larger the interval, the fewer points are taken, the faster the processing time is, and the accuracy will be slightly lost. The solution of this embodiment is that the size of the interval can be flexibly set to balance the relationship between accuracy and speed. This is because in industrial detection, image acquisition is often very dense, and usually it is not necessary to select all points for fitting.

[0086] Step 504: fit multiple support points to obtain a reference model corresponding to the image unit.

[0087] Since the benchmark model is obtained by fitting based on multiple support points, and the benchmark model corresponds to the image unit.

[0088] In some embodiments, fitting is performed based on multiple support points to obtain a reference model corresponding to the image unit, including:

[0089] Fitting is performed according to a plurality of supporting points in each benchmark fitting region to obtain a benchmark model corresponding to each image unit.

[0090] Therefore, mapping is performed based on the respective reference models corresponding to each image unit, so as to estimate the mapped image unit of each image unit from the local angle of the target part in the reference fitting area and analyze the defects of the target part.

[0091] In some embodiments, fitting is performed based on multiple support points to obtain a reference model corresponding to the image unit, including:

[0092] Fitting is performed according to multiple supporting points of each image unit to obtain a reference model corresponding to each image unit; weighted processing is performed on the reference model corresponding to each image unit to obtain a reference model commonly corresponding to all image units.

[0093] Therefore, averaging, standardization and other processing are performed based on the reference model corresponding to each image unit, so as to estimate the mapped image unit of each image unit from the overall perspective of the target part and analyze the defects of the target part.

[0094] It can be seen that in this embodiment, multiple support points are selected according to the preset step size. By adjusting the numerical value of the preset step size, it is possible to reduce the support points required for fitting and improve the fitting efficiency on the basis of controlling the accuracy and efficiency of the benchmark model. At the same time, by selecting multiple points along the target direction, the support point selection rules can be adaptively adjusted according to the point selection requirements to ensure the fitting efficiency.

[0095] In some embodiments, the target direction includes at least one of a first direction and a second direction of the reference fitting region. The first direction and the second direction are at least two different directions set for the reference fitting region, and are used to locate and select points at different positions of the reference fitting region. Fitting is performed based on multiple points in the reference fitting region to obtain a reference model, including:

[0096] According to a preset step length, along at least one of the first direction and the second direction, multiple points in the reference fitting area are selected to obtain multiple support points;

[0097] Curve fitting is performed on multiple support points to obtain a benchmark model corresponding to the image unit.

[0098] In some embodiments, curve fitting is performed on multiple support points to obtain a reference model corresponding to the image unit, including:

[0099] According to the initial curve model, determine the initial prediction value corresponding to the support point;

[0100] The residual is calculated based on the initial prediction value and the support point until the residual value is minimized, thereby obtaining a reference model corresponding to the image unit;

[0101] The residual is the sum of squares of the difference between the initial prediction value and the pixel value of each weighted support point. Thus, the moving least squares method is used for fitting.

[0102] In some embodiments, curve fitting is performed on multiple support points to obtain a reference model corresponding to the image unit, including:

[0103] Determine the weight of each support point according to the distance between the unit reference point and the multiple support points;

[0104] According to the weight of each support point, each support point is weighted to obtain a weighted support point;

[0105] According to the initial curve model, determine the initial prediction value corresponding to the weighted support point;

[0106] The residual is calculated based on the initial prediction value and the weighted support point until the residual value is minimized, thereby obtaining a reference model corresponding to the image unit;

[0107] The residual is the sum of squares of the difference between the initial prediction value and the pixel value of each weighted support point. Thus, the moving least squares method is used for fitting.

[0108] It should be understood that in the process of curve fitting, the data in the image unit is in the form of a two-dimensional depth map to improve processing efficiency. Correspondingly, if the initial depth image exists in the form of a three-dimensional point cloud, it needs to be converted into a two-dimensional depth map, and then the steps for obtaining multiple support points and the steps for obtaining the reference model corresponding to the image unit are performed.

[0109] It can be seen that in this embodiment, at least one of the first direction and the second direction is used to select points, so that the direction of point selection is adjustable and controllable; on this basis, curve fitting is performed according to the distance, and a fitting process of a univariate multivariate equation can be formed, and the amount of calculation required for the calculation is relatively small, so as to save time and improve the processing speed.

[0110] In some embodiments, the target direction includes at least one of a first direction and a second direction of the reference fitting area.

[0111] According to a preset step length, along at least one of the first direction and the second direction, multiple points in the reference fitting area are selected to obtain multiple support points, including:

[0112] When the target direction is the first direction, multiple support points are selected from the points arranged in the second direction along the first direction according to a preset step length; or

[0113] When the target direction is the second direction, multiple support points are selected from the points arranged in the first direction along the second direction according to a preset step length.

[0114] In some embodiments, according to a preset step length, along the first direction, multiple support points are selected from points arranged in the second direction, including:

[0115] According to the preset step length in the horizontal direction, multiple support points are selected from each row of points arranged in sequence in the horizontal direction along the vertical direction. Therefore, for the preset step length set in the horizontal direction, corresponding support points are selected from the points in each row to adapt to the point selection interval in the horizontal direction. Therefore, for the commonly used collection process, the adapted support point collection process is determined to ensure reliability.

[0116] Specifically, the data (columns / rows) are divided according to the first direction, the data (columns / rows) are arranged in the second direction, and support points are selected along the second direction with a preset step size. If a straight line is to be fitted along the row direction, points are taken at intervals for a group of data in the row direction.

[0117] Specifically, the points arranged in the second direction are screened according to the preset step length in the first direction; when the screened points reach the preset number, the preset number of points are used as multiple support points. Thus, the number of support points is controlled by the preset number for more accurate adjustment. Among them, multiple support points can be selected from the columns of points arranged in sequence in the vertical direction along the horizontal direction according to the preset step length in the vertical direction. Thus, for the preset step length set in the vertical direction, the corresponding support points are selected from the points in each column to adapt to the point selection interval in the vertical direction. Thus, for the commonly used acquisition process, the adapted support point acquisition process is determined to ensure reliability. Specifically, the number of support points is controlled by the preset number for more accurate adjustment.

[0118] Specifically, there are n rows and m columns of pixel points in an image unit. In the traditional technology, each point needs to be fitted once, which requires fitting n×m times; while in the present embodiment, when fitting a local curve, therefore, when fitting in a single direction, it only needs to fit n or m times; for example: when selecting points and simulating along n rows in sequence, it needs to fit n times, and when selecting points and simulating along m columns in sequence, it needs to fit m times; and when selecting the first direction and the second direction for fitting at the same time, it only needs to fit m+n times.

[0119] It can be seen that in this embodiment, in the first direction and the second direction, a corresponding support point is selected by determining a direction separately, so that the number of support points is relatively small, and the processing efficiency is relatively high. Moreover, when the surface trend is continuous in one direction and discontinuous in another direction, the directional curve fitting in a single direction is fitted, and the fitted benchmark model is more accurate.

[0120] In some embodiments, the target direction is consistent with the acquisition direction of the initial depth image.

[0121] The acquisition direction refers to the direction in which each point in the initial depth image is acquired. Specifically, the acquisition direction may be a scanning direction or a shooting direction of a 3D camera.

[0122] Specifically, when the initial depth image is collected along the first direction, the target direction is the first direction; when the initial depth image is collected along the second direction, the target direction is the second direction. Therefore, the collection direction of the initial depth image is consistent with the selection direction of the support point, so the support point is selected for the dense data in the collection direction to improve the accuracy.

[0123] Specifically, when the initial depth image is collected along the first direction, the target direction is the second direction; when the initial depth image is collected along the second direction, the target direction is the first direction. Therefore, when there is a deviation between the first direction and the second direction, the collection direction of the initial depth image is consistent with the direction of selecting the support point, so the support point is selected for data outside the collection direction to improve efficiency.

[0124] It can be seen that in this embodiment, the target direction and the collection direction are matched, and the target direction can be selected specifically to achieve the corresponding effect, thereby balancing accuracy and efficiency.

[0125] In some embodiments, the target direction includes a first direction and a second direction of the reference fitting area;

[0126] The benchmark model is obtained by fitting multiple points in the benchmark fitting area, including:

[0127] According to the preset step length, multiple points in the reference fitting area are selected along the target direction to obtain the support points in the first direction and the support points in the second direction.

[0128] Correspondingly, curve fitting is performed on multiple support points to obtain the benchmark model corresponding to the image unit, including:

[0129] Performing curve fitting on a plurality of support points in a first direction to obtain a first reference model corresponding to the image unit;

[0130] Performing curve fitting on a plurality of support points in the second direction to obtain a second reference model corresponding to the image unit;

[0131] Correspondingly, according to the benchmark model, the image unit is mapped to obtain the mapped image unit, including:

[0132] According to the first reference model, the image unit is mapped to obtain a first mapping value;

[0133] According to the second reference model, the image unit is mapped to obtain a second mapping value;

[0134] The first mapping value and the second mapping value are numerically fused to obtain a mapped image unit.

[0135] Specifically, for the support points in the first direction and the support points in the second direction, the acquisition process includes: selecting multiple points in the benchmark fitting area along the first direction according to the preset step size of the first direction to obtain multiple support points in the first direction; selecting multiple points in the benchmark fitting area along the second direction according to the preset step size of the second direction to obtain multiple support points in the second direction.

[0136] Specifically, the process of obtaining the support points in the first direction includes:

[0137] According to the preset step length in the horizontal direction, multiple support points are selected from the rows of points arranged in sequence in the horizontal direction along the vertical direction. Or,

[0138] The points arranged in the second direction are screened according to the preset step length in the first direction; when the screened points reach a preset number, the preset number of points are used as multiple supporting points.

[0139] Specifically, the process of obtaining the supporting points in the second direction includes:

[0140] According to the preset step length in the vertical direction, multiple support points are selected from the columns of points arranged in sequence in the vertical direction along the horizontal direction. Or,

[0141] The points arranged in the first direction are screened according to the preset step length in the second direction; when the screened points reach a preset number, the preset number of points are used as multiple supporting points.

[0142] The first reference model is a curve model, and is obtained by fitting multiple support points in a first direction; the second reference model is another curve model, and is obtained by fitting multiple support points in a second direction. Since the first reference model and the second reference model are obtained by fitting different data, they can estimate the theoretical pixel value of the image unit from different angles. Specifically, the first reference model and the second reference model both include pixel value weighting, and may also include pixel position weighting.

[0143] The first mapping value is obtained by weighting the pixel points in the image unit according to the first reference model. The second mapping value is obtained by weighting the pixel points in the image unit according to the second reference model.

[0144] In some embodiments, numerically fusing the first mapping value and the second mapping value to obtain a mapped image unit includes:

[0145] In the image unit, determine the pixel points to be fused corresponding to the positions;

[0146] Averaging processing is performed according to the first mapping value and the second mapping value corresponding to each point to be fused to obtain a mapped image unit.

[0147] The pixel points to be fused that correspond to each other in position refer to the pixel points obtained by mapping the same position; or the pixel points obtained by mapping the positions that have corresponding relationship expressions. The pixel points to be fused may be each pixel point or a set of multiple pixel points.

[0148] In some embodiments, numerically fusing the first mapping value and the second mapping value to obtain a mapped image unit includes:

[0149] In the image unit, determine the pixel points to be fused corresponding to the positions;

[0150] Screening is performed according to the first mapping value and the second mapping value corresponding to each point to be fused;

[0151] The mapped image unit is determined according to the filtered mapping value.

[0152] Therefore, these two types of data are screened to obtain corresponding mapped image units according to different requirements of the target parts.

[0153] It can be seen that in this embodiment, points are selected according to both the first direction and the second direction, and then the support points in the two target directions are fitted respectively to obtain two sets of curve models; then the two sets of curve models are mapped and numerically fused respectively, so as to ensure that the accuracy of the mapped image unit is relatively high through the fusion of the two sets of curves.

[0154] In some embodiments, the target direction includes the surface direction indicated by the first direction and the second direction of the reference fitting area; the image unit includes a unit reference point, which is at least one pixel point in the image unit, which is used to represent the depth information of the image unit relative to the camera, so as to more finely control the accuracy and efficiency of subsequent fitting.

[0155] The benchmark model is obtained by fitting multiple points in the benchmark fitting area, including:

[0156] According to the preset step length, multiple points in the reference fitting area are selected along the surface direction to obtain multiple support points;

[0157] The unit reference point and multiple support points are fitted with a surface to obtain a reference model corresponding to the image unit.

[0158] The surface direction has directions indicated by the first direction and the second direction respectively. Specifically, the first direction is the horizontal direction, the second horizontal direction is the vertical direction, and the surface direction is a direction having both horizontal and vertical dimensions. Specifically, the surface direction is a direction adaptively adjusted according to the change rule of the reference fitting area.

[0159] In some embodiments, performing surface fitting on multiple support points to obtain a reference model corresponding to the image unit includes:

[0160] Determine the weight of each support point according to the Euclidean distance between the unit reference point and the plurality of support points;

[0161] According to the weight of each support point, each support point is weighted to obtain a weighted support point;

[0162] According to the initial surface model, determine the initial prediction value corresponding to the weighted support point;

[0163] The residual is calculated based on the initial prediction value and the weighted support point until the residual value is minimized, thereby obtaining a reference model corresponding to the image unit;

[0164] The residual is the square sum of the difference between the initial prediction value and the pixel value of each weighted support point. Therefore, the moving least squares method is used for fitting.

[0165] It should be understood that in the process of surface fitting, the data in the image unit is in the form of a three-dimensional point cloud to improve the accuracy when the surface of the target part is a curved surface. Correspondingly, if the initial depth image exists in the form of a two-dimensional depth map, it needs to be converted into a three-dimensional point cloud, and then the steps for obtaining multiple support points and the steps for obtaining the reference model corresponding to the image unit are performed.

[0166] It can be seen that in this embodiment, the curved surface direction is constructed according to the first direction and the second direction, so that the present solution can select multiple support points along the corresponding curved surface to improve the accuracy when the surface of the target part is a curved surface.

[0167] In some embodiments, fitting is performed based on multiple support points to obtain a reference model corresponding to the image unit, including:

[0168] The weight of each support point is determined according to the distance between the unit reference point and multiple support points; the distance and weight are negatively correlated;

[0169] According to the weights, multiple support points are weighted to obtain multiple weighted support points;

[0170] Fitting is performed based on multiple weighted support points to obtain a benchmark model corresponding to the image unit.

[0171] Since the distance and weight are negatively correlated, the smaller the distance between the support point and the unit reference point, the greater its weight; the larger the distance between the support point and the unit reference point, the smaller its weight. Specifically, when the unit reference point is the center point of the image unit, the fitting equation is calculated only once. At this time, the closer the point is to the center, the greater the weight, and the farther the point is, the lower the weight.

[0172] The weighted support point is a pixel point at a corresponding position to the support point, and the pixel value is adjusted according to the weight. Specifically, when the support point is at the coordinate (x, y), the corresponding weighted support point is also at the coordinate (x, y).

[0173] In some embodiments, weighted processing is performed on multiple support points according to weights to obtain multiple weighted support points, including: weighted processing is performed on pixel values ​​of multiple support points according to weights to obtain multiple weighted support points.

[0174] In some embodiments, fitting is performed based on a plurality of weighted support points to obtain a reference model corresponding to the image unit, including:

[0175] According to the initial benchmark model, determine the initial prediction value corresponding to the weighted support point;

[0176] The residual is calculated based on the initial prediction value and the weighted support point until the residual value is minimized, thereby obtaining a reference model corresponding to the image unit;

[0177] Among them, the residual is the sum of squares of differences, which is the difference between the initial prediction value and the pixel value of each weighted support point; when the initial benchmark model is a curve model, the benchmark model is a curve model; when the initial benchmark model is a surface model, the benchmark model is a surface model.

[0178] In some embodiments, a neural network model may be used to perform fitting based on a plurality of weighted support points to obtain a reference model corresponding to the image unit.

[0179] It can be seen that in this embodiment, multiple support points are negatively correlated with the weight of each support point, and weighted processing is performed according to the weight to obtain multiple weighted support points, so that the support points close to the unit reference point contribute more to the benchmark model; finally, fitting is performed based on the multiple weighted support points, so that the benchmark model can more accurately determine the theoretical mapped image unit.

[0180] In some embodiments, the center point of the image unit is the unit reference point, and multiple points in the reference fitting area are fitted with the center point to form a reference model.

[0181] The center point is a pixel point located at the center of the area contained in the image unit. Specifically, when the image unit is a rectangle, the center point is the intersection of the diagonals of the rectangle; specifically, when the image unit is a circle, the center point is the center of the circle.

[0182] It can be seen that in this embodiment, using the center point as the unit reference point can reduce the number of fitting times and can more accurately represent the depth information of the entire image unit relative to other image units to ensure accuracy.

[0183] In some embodiments, the plurality of points in the reference fitting region are points in the same row or column as the image unit.

[0184] Specifically, a plurality of points in the reference fitting area are support points, and the support points are points in the same row or column as the image unit.

[0185] Specifically, the multiple points in the reference fitting region are points in the reference fitting region that are in the same row as any point in the image unit. Specifically, the multiple points in the reference fitting region are points in the reference fitting region that are in the same column as any point in the image unit. Specifically, the multiple points in the reference fitting region are points in the reference fitting region that are in the same column as any point in the image unit, and points in the same row as any point in the image unit. Specifically, the reference fitting region is the region after the image unit is expanded along four directions, that is, the points in the image unit are a subset of the reference fitting region.

[0186] It can be seen that in this embodiment, in the reference fitting area expanded by the image unit, corresponding points are selected for fitting according to the position of the image unit, so as to reduce the amount of data required for fitting and ensure the fitting efficiency.

[0187] In some embodiments, the image units are determined by dividing the filtered initial depth image according to a preset size; that is, determining the image units into which the initial depth image is divided includes: after filtering the initial depth image, determining the image units into which the initial depth image is divided according to the preset size in the filtered initial depth image.

[0188] The filtered initial depth image is a denoised depth image. Since the defect detection process necessarily involves the acquisition process of the initial depth image, which may contain invalid points or abnormal points, it is necessary to perform filtering before dividing the corresponding image units.

[0189] Specifically, for the filtered initial depth image, the filtering methods that can be used include, but are not limited to, spatial domain filtering processing such as mean filtering and Gaussian filtering, and frequency domain filtering processing such as low-pass filtering and band-pass filtering.

[0190] In some embodiments, the preset size is negatively correlated with the accuracy of defect detection, and the preset size is positively correlated with the efficiency of defect detection.

[0191] The preset size is used to determine the range of the image unit. The larger the preset size, the more pixels there are in each image unit, and the smaller the preset size, the smaller the pixels there are in each image unit. Specifically, the preset size can be a side length, a radius, or other data set for the target part.

[0192] In some embodiments, determining image units into which the initial depth image is divided according to a preset size includes: determining grid-shaped image units into which the initial depth image is divided according to a preset side length.

[0193] In some embodiments, determining the image units into which the initial depth image is divided according to a preset size includes: in response to an instruction for representing a high precision requirement, determining the image units into which the initial depth image is divided according to a first preset size; in response to an instruction for representing a high efficiency requirement, determining the image units into which the initial depth image is divided according to a second preset size; wherein the first preset size is smaller than the second preset size. Specifically, the image unit is a grid, and for scenes with higher precision requirements, a smaller grid size can be selected as the preset size, while for scenes with higher efficiency requirements, a larger grid size can be selected as the preset size.

[0194] It can be seen that, on the one hand, in this embodiment, the initial depth image is first filtered to complete the smoothing and denoising of the initial depth image, to prevent some invalid points or abnormal points from participating in the division of image units and the calculation of benchmark model fitting, resulting in the fitting of a model that does not meet expectations. Therefore, filtering the initial depth image helps to improve the accuracy of the results mapped by the benchmark model, so as to more accurately determine defects. On the other hand, since the number of image units determines the number of fitted benchmark models, when the amount of data occupied by the fitting process during defect detection is too large, the accuracy and efficiency of defect detection can be controlled separately by preset sizes. The image units are divided according to preset sizes, and the accuracy and efficiency of defect detection can be adjusted by preset sizes to balance the two.

[0195] In some embodiments, defect detection is performed on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part, including:

[0196] Perform filtering processing according to each mapped image unit to obtain a filtered depth image;

[0197] According to the image difference between the mapped image unit and the initial depth image, defect detection is performed on the initial depth image to obtain the defect detection result of the target part.

[0198] The filtered depth image includes each mapped image unit after filtering. Specifically, the filtered depth image can be composed according to the position of each mapped image unit in the initial depth image. Specifically, the filtered depth image is a set, and this set includes each mapped image unit after filtering. The filtered depth image can be in the form of a surface model 3D image or a two-dimensional depth map.

[0199] In some embodiments, filtering is performed on each mapped image unit to obtain a filtered depth image, including:

[0200] Determine the positional relationship of filtering processing of each mapped image unit according to the position of the image unit divided by each mapped image unit;

[0201] According to the positional relationship, the pixel values ​​of each mapped image unit are filtered to obtain a filtered depth image.

[0202] In some embodiments, filtering is performed on each mapped image unit to obtain a filtered depth image, including:

[0203] Perform mean filtering or Gaussian filtering on each mapped image unit to obtain a filtered depth image.

[0204] In some embodiments, performing defect detection on the initial depth image according to the filtered depth image includes:

[0205] According to the image difference between the mapped image unit and the initial depth image, the defects represented by the initial depth image are detected.

[0206] In some embodiments, detecting defects represented by the initial depth image according to an image difference between the mapped image unit and the initial depth image includes:

[0207] The mapped image unit is differentially processed with the initial depth image to obtain a distance map; and the concave-convex defects with distance exceeding the index are extracted according to the distance size in the distance map.

[0208] In some embodiments, detecting defects represented by the initial depth image according to an image difference between the mapped image unit and the initial depth image includes:

[0209] At each pixel position, a difference processing is performed on the pixel value between the mapped image unit and the initial depth image to obtain a difference value;

[0210] Defects represented by the initial depth image are detected according to the interval in which the difference value of each pixel position lies.

[0211] It can be seen that in this embodiment, on the one hand, since the reference model is obtained by fitting each image unit separately, there may be connection problems such as sawtooth between different mapped image units. By filtering each mapped image unit, a filtered depth image can be formed, making the image smoother, so as to improve efficiency while eliminating the slicing phenomenon between the introduced cells, so as to more accurately detect the defects contained in the initial depth image. On the other hand, through image differences, the defects in the initial depth image can be quantified, so as to more accurately quantify the defects in the target part, accurately judge whether the target part has a concave or convex problem, and even determine the degree of concave or convex.

[0212] In a specific embodiment, the method comprises:

[0213] In the first step, the 3D camera collects the initial depth image;

[0214] The second step is to perform preprocessing such as filtering on the initial depth image to smooth and remove noise;

[0215] The third step is to divide the initial depth image into grids according to the detection accuracy and speed requirements; each grid is an image unit;

[0216] The fourth step is to traverse each grid and execute steps 1)-4) to rebuild the benchmark model:

[0217] 1) First, expand a certain range around the image cell to form a benchmark fitting area block;

[0218] 2) Take points at intervals within the benchmark fitting area to obtain support points;

[0219] 3) Fit according to the support points to obtain the benchmark model;

[0220] 4) Calculate the projection of the points in the detection area on the reference model, map it back to the overall depth data, and obtain the mapped image unit; the detection area is a subset of the image unit, and the image unit is the grid result of the detection area, and each image unit can be a square.

[0221] The fifth step is to filter the fitted mapped image units to eliminate jagged edges of some image units and make them smooth, thereby obtaining a filtered depth image.

[0222] In the sixth step, defect detection is performed on the initial depth image based on the filtered depth image.

[0223] Among them, the first step and the third step correspond to step 202 of the above embodiment, 1)-3) of the fourth step correspond to step 204 of the above embodiment, 4) of the fourth step corresponds to step 206 of the above embodiment, and the sixth step corresponds to step 208 of the above embodiment.

[0224] In the first step, the 3D camera collects the initial depth image, which includes: In industrial inspection scenarios, the 3D camera used is a 3D line scan laser camera or a structured light camera. The output 3D data is usually a depth map, which can be converted into a 3D structure point cloud, that is, the structural relationship between adjacent points is retained.

[0225] Among them, the depth map and the structure point cloud are considered equivalent, both of which are 3D data and can be converted to each other. The 3D data output by the 3D camera can be converted into a depth map, which is more convenient for storage and reading and writing through more 2D processing tools. Of course, you can also choose to output point cloud files such as .ply. In addition, the depth map can also be conveniently processed using some 2D image processing algorithms. However, some operations, such as calculating surface equations, require converting the data back to 3D points (x, y, z), which is not supported by the depth map. Therefore, it needs to be converted into a 3D structure point cloud.

[0226] The second step is to perform preprocessing such as filtering on the initial depth image to smooth and remove noise, including: selecting conventional mean filtering, Gaussian filtering and other methods for filtering. The main function is to remove some noise from the original data and prevent some invalid points or abnormal points from participating in the calculation of the benchmark model fitting, resulting in a model that does not meet expectations.

[0227] The third step is to divide the depth image into grids according to the accuracy and speed requirements, including: dividing the detection area in the depth image into grids. The size of the grid division can be adjusted according to the accuracy and processing speed requirements. For scenes with high accuracy requirements, a smaller grid size can be selected, while for scenes with high efficiency requirements, a larger grid size can be selected.

[0228] The purpose of dividing the grid is to reduce the time spent on calculation. Traditional moving least squares is for surface reconstruction. For each point, points within a certain distance of the neighborhood are selected for benchmark surface fitting. In this embodiment, since the surface trends of the neighborhoods of points within a certain range are approximate, for each point in the grid, only the neighborhood points corresponding to the center point need to be fitted once, and the resulting model can be used for other points. This processing method can greatly reduce the time spent on the algorithm. For example, when the size of the grid is 8×8, the time used for model fitting is reduced by nearly 98% when processing an area of ​​the size of a grid.

[0229] Step 4: Traverse the grid to reconstruct the benchmark model. This step mainly processes each grid divided in step 3. The specific process is as follows:

[0230] First, for each grid, a certain range is expanded as the fitting support domain (block) of the benchmark model. The expanded range can be adjusted according to the actual application scenario. For example, for a surface with gentle changes, a larger expansion size can be set, and for a surface with significant changes, a smaller expansion size can be set.

[0231] Then, points in the support domain are selected as support points to fit the benchmark model. In order to further improve the computational efficiency, the number of points used for model fitting can be reduced by presetting the step size and taking points at fixed intervals.

[0232] Next, use the support points to fit the benchmark model. For the fitted model, according to the actual application scenario, and the requirements for detection accuracy and speed, you can choose curve fitting and surface fitting; curve fitting includes point selection in a single target direction and point selection in multiple target directions; point selection in a single target direction includes support point selection for the X-direction curve and the Y-direction curve; and point selection in multiple target directions includes support point selection for the XY-direction curve.

[0233] For the X-direction curve model, the specific operation is to divide the block into columns of data points along the X-direction. For each column of data, points are taken as support points at fixed intervals, where each support point is a two-dimensional point, the horizontal coordinate u is the y value of the 3D data, and the vertical coordinate v is the z value of the 3D data. Then substitute the support points into the moving least squares method to fit the corresponding curve model, and calculate the projection of each point in the column on the curve to obtain the model reference point of the point. By processing each column in the block in the same way, the model reference point corresponding to the detection point in the entire grid can be obtained.

[0234] Similarly, for the Y-direction curve model, the support domain is divided into rows of data points along the Y direction. For each row of data, points are taken at fixed intervals, and then curve fitting is performed. After obtaining the curve equation, the projection of the points in the column in the grid on the curve is calculated to obtain the model reference point.

[0235] For the XY direction curve model, support points are selected along the X direction and the Y direction to fit the curve. Then, for each detection point in the grid, its projection on the corresponding two curve models is calculated, and the final model result is the average of the projection points in the two directions.

[0236] For the surface model, points are taken along the X and Y directions at fixed intervals, and then the moving least squares method is used for surface fitting. The projection of the detection area on the surface is then calculated as the fitting model points.

[0237] The fitting method uses the moving least squares method. The idea of ​​the moving least squares method is that the dependent variable y at the independent variable x is only affected by the nodes near x. Therefore, for surface reconstruction, the surface trend at a point is only related to the points in its neighborhood area, and has nothing to do with other points outside the area.

[0238] It is worth noting that since the model is only fitted once for all detection points in the grid, when using the moving least squares method to calculate the weights of neighborhood points, the distance from the neighborhood point to the center point in the grid is calculated as the weight, and all support points are weighted to obtain weighted support points; and the least squares method is performed on the weighted support points.

[0239] From the perspective of fitting and point selection processing speed, X\Y single direction is faster than XY bidirectional, and XY bidirectional is faster than surface. In addition, the curve equation is a one-variable multi-time equation, while the surface is a two-variable multi-time equation. In terms of calculation time, the surface will be much slower.

[0240] From the perspective of processing accuracy, when the surface trend of the target part is continuous in one direction and discontinuous data distribution in another direction, the benchmark model fitted by X\Y single-direction curve fitting in a single direction is more accurate. When the surface trend of the target part is continuous in multiple directions and discontinuous data distribution in another direction, the benchmark model fitted by X\Y single-direction curve fitting in a single direction is more accurate.

[0241] The fifth step is to smooth the fitted depth map, including: using a mean filter or a Gaussian filter to smooth the fitted depth map data, thereby obtaining a smooth surface model 3D map, thereby eliminating the aliasing problem between adjacent image units.

[0242] The sixth step is to perform defect detection on the initial depth image according to the filtered depth image, including: obtaining a distance map from the original point cloud to the filtered depth image by subtracting the filtered depth image from the original depth image.

[0243] This has at least two effects:

[0244] On the one hand, the calculation time is greatly reduced, so that the method can be applied to real-time defect detection. When calculating the benchmark model, this embodiment can freely divide the image unit and the benchmark fitting area according to the actual scene requirements, and for all points in the image unit, only the expanded benchmark fitting area is used to calculate the model fitting once to achieve a balance between efficiency and accuracy. At the same time, instead of selecting all points in the benchmark fitting area for calculation, the point taking step size can be freely set, and points can be taken at certain intervals to reduce the amount of calculation. And while improving efficiency, in order to eliminate the slicing phenomenon between the introduced image units, a filtering operation is added to perform a smooth transition.

[0245] On the other hand, through the selection of the model, different fitting models can be switched for different scenes to balance the accuracy and speed requirements of the actual scene. Since the surface equation has two unknowns, and the curve equation has only one unknown, it takes more time to calculate the surface equation than to calculate the curve equation during the fitting calculation. To this end, the present embodiment provides different fitting model calculation methods: X-direction curve, Y-direction curve, XY bidirectional curve and surface. For scenes with high timeliness requirements, you can choose to fit the curves in the X and Y directions, and the XY bidirectional curve is a compromise. In addition, when the surface trend is continuous in one direction and discontinuous in another direction, fitting the X-direction curve or the Y-direction curve has a better fitting effect than fitting the surface.

[0246] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0247] Based on the same inventive concept, the embodiment of the present application also provides a defect detection device for a target part. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the defect detection device for one or more target parts provided below can refer to the limitations of the defect detection method for the target part above, and will not be repeated here.

[0248] like Figure 6 As shown, an embodiment of the present application provides a defect detection device for a target part, comprising:

[0249] An acquisition module 602 is used to acquire an initial depth image of the target part and determine image units into which the initial depth image is divided;

[0250] A fitting module 604 is used to determine a reference fitting area extending from the image unit according to the extension range, select a plurality of points in the reference fitting area along the target direction, and perform fitting according to the plurality of points in the reference fitting area to obtain a reference model;

[0251] A mapping module 606, configured to map the image unit according to the reference model to obtain a mapped image unit;

[0252] The detection module 608 is used to perform defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part.

[0253] In some embodiments, the target direction includes at least one of a first direction and a second direction of the reference fitting region; in terms of fitting according to multiple points in the reference fitting region to obtain the reference model, the fitting module 604 is specifically used to:

[0254] When the target direction is the first direction, multiple support points are selected from the points arranged in the second direction along the first direction according to a preset step length; or

[0255] When the target direction is the second direction, selecting a plurality of support points from the points arranged in the first direction along the second direction according to a preset step length;

[0256] Curve fitting is performed on multiple support points to obtain a benchmark model corresponding to the image unit.

[0257] In some embodiments, the target direction is consistent with the acquisition direction of the initial depth image.

[0258] In some embodiments, the target direction includes a first direction and a second direction of the reference fitting region; in selecting a plurality of points in the reference fitting region along the target direction, the fitting module 604 is specifically configured to:

[0259] According to a preset step length, multiple points in the reference fitting area are selected along the target direction to obtain support points in the first direction and support points in the second direction;

[0260] In terms of performing curve fitting on multiple support points to obtain a reference model corresponding to the image unit, the fitting module 604 is specifically used for:

[0261] Performing curve fitting on a plurality of support points in a first direction to obtain a first reference model corresponding to the image unit;

[0262] Performing curve fitting on a plurality of support points in the second direction to obtain a second reference model corresponding to the image unit;

[0263] Correspondingly, in terms of mapping the image unit according to the reference model to obtain the mapped image unit, the mapping module 606 is specifically used for:

[0264] According to the first reference model, the image unit is mapped to obtain a first mapping value;

[0265] According to the second reference model, the image unit is mapped to obtain a second mapping value;

[0266] The first mapping value and the second mapping value are numerically fused to obtain a mapped image unit.

[0267] In some embodiments, the target direction includes the first direction of the reference fitting region and the surface direction indicated by the second direction, and the image unit includes a unit reference point; in performing fitting according to multiple points in the reference fitting region to obtain the reference model, the fitting module 604 is specifically used to:

[0268] According to the preset step length, multiple points in the reference fitting area are selected along the surface direction to obtain multiple support points;

[0269] The unit reference point and multiple support points are fitted with a surface to obtain a reference model corresponding to the image unit.

[0270] In some embodiments, the plurality of points in the reference fitting region are points in the same row or column as the image unit; and / or,

[0271] The image unit is determined by dividing the filtered initial depth image according to a preset size; and / or,

[0272] The center point of the image unit is the unit reference point, and multiple points in the benchmark fitting area and the center point are fitted to form a benchmark model.

[0273] In some embodiments, in terms of performing defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part, the detection module 608 is specifically used to:

[0274] Perform filtering processing according to each mapped image unit to obtain a filtered depth image;

[0275] According to the image difference between the filtered depth image and the initial depth image, defect detection is performed on the initial depth image to obtain the defect detection result of the target part.

[0276] Each module in the above-mentioned defect detection device for target parts can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0277] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, the steps in the defect detection method of the target part described above are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0278] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0279] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0280] In some embodiments, the storage medium may be a computer-readable storage medium; Figure 8 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0281] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0282] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0283] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0284] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0285] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting defects of a target part, characterized in that: include: Acquire an initial depth image of the target part, and determine image units into which the initial depth image is divided; Determine a reference fitting area extending from the image unit according to the extension range, select a plurality of points in the reference fitting area along a target direction, and perform fitting according to the plurality of points in the reference fitting area to obtain a reference model; According to the reference model, mapping the image unit to obtain a mapped image unit; According to the mapped image unit, defect detection is performed on the initial depth image to obtain a defect detection result of the target part.

2. The method according to claim 1, characterized in that The target direction includes at least one of a first direction and a second direction of the reference fitting area; The selecting a plurality of points in the reference fitting area along the target direction and performing fitting according to the plurality of points in the reference fitting area to obtain a reference model includes: When the target direction is the first direction, selecting a plurality of support points from the points arranged in the second direction along the first direction according to a preset step length; or When the target direction is the second direction, selecting a plurality of support points from the points arranged in the first direction along the second direction according to a preset step length; Curve fitting is performed on the multiple support points to obtain a reference model corresponding to the image unit.

3. The method according to claim 2, characterized in that The target direction is consistent with the acquisition direction of the initial depth image.

4. The method according to claim 2, characterized in that: The target direction includes a first direction and a second direction of the reference fitting area, The selecting a plurality of points in the reference fitting area along the target direction comprises: According to a preset step length, multiple points in the reference fitting area are selected along the target direction to obtain support points in the first direction and support points in the second direction; The performing curve fitting on the plurality of support points to obtain a reference model corresponding to the image unit includes: Performing curve fitting on a plurality of supporting points in the first direction to obtain a first reference model corresponding to the image unit; Performing curve fitting on a plurality of supporting points in the second direction to obtain a second reference model corresponding to the image unit; Mapping the image unit according to the reference model to obtain a mapped image unit includes: Mapping the image unit according to the first benchmark model to obtain a first mapping value; Mapping the image unit according to the second reference model to obtain a second mapping value; The first mapping value and the second mapping value are numerically fused to obtain a mapped image unit.

5. The method according to claim 1, characterized in that The target direction includes the first direction and the surface direction indicated by the second direction of the reference fitting area, and the image unit includes a unit reference point; The step of selecting a plurality of points in the reference fitting area along the target direction and performing fitting according to the plurality of points in the reference fitting area to obtain a reference model includes: According to a preset step length, multiple points in the reference fitting area are selected along the surface direction to obtain multiple support points; Surface fitting is performed on the unit reference point and the plurality of support points to obtain a reference model corresponding to the image unit.

6. The method according to claim 1, characterized in that The plurality of points in the reference fitting area are points in the same row or column as the image unit; and / or, The image unit is determined by dividing the filtered initial depth image according to a preset size; and / or, The center point of the image unit is the unit reference point, and multiple points in the reference fitting area and the center point are fitted to form the reference model.

7. The method according to claim 1, characterized in that The step of performing defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part includes: Performing filtering processing according to each of the mapped image units to obtain a filtered depth image; According to the image difference between the filtered depth image and the initial depth image, defect detection is performed on the initial depth image to obtain a defect detection result of the target part.

8. A defect detection device for a target part, characterized in that: include: An acquisition module, used to acquire an initial depth image of a target part and determine image units into which the initial depth image is divided; A fitting module, used to determine a reference fitting area extending from the image unit according to the extension range, select a plurality of points in the reference fitting area along a target direction, and perform fitting according to the plurality of points in the reference fitting area to obtain a reference model; A mapping module, used for mapping the image unit according to the reference model to obtain a mapped image unit; A detection module is used to perform defect detection on the initial depth image according to the mapped image unit to obtain a defect detection result of the target part.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images