Data acquisition method and system, electronic equipment, storage medium and program product

Abstract

The invention provides a data acquisition method and system, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring first sparse point cloud data of a free-form surface of a material; according to the first sparse point cloud data and a sensing area of a first line sensor, determining a moving path of the material along with the height change of the free-form surface; and moving the material based on the moving path, and controlling the first line sensor to scan the surface of the free-form surface to obtain scanning image data of different heights of the surface of the free-form surface. According to the method, image data of different heights of the free-form surface can be scanned, and complete image data of the free-form surface can be obtained.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning measurement technology, and in particular to a data acquisition method, system, electronic equipment, storage medium and program product. Background Art

[0002] Free-form surface data acquisition is used to collect image data of irregular geometric feature surfaces. In industrial material manufacturing, for materials with free-form surfaces, free-form surface data acquisition methods can be used to collect image data of the material surface for surface defect detection.

[0003] In the related art, the data acquisition system can control the horizontal movement of the material, and during the movement of the material, the free-form surface of the material is scanned by a line sensor fixed above the material to obtain image data of the free-form surface scanned by the line sensor. Since the sensing area of ​​the line sensor is limited, when the height change of the free-form surface exceeds the sensing area of ​​the line sensor, the line sensor cannot scan the part of the free-form surface that exceeds the sensing area of ​​the line sensor, and the image data of the free-form surface collected is incomplete. Summary of the invention

[0004] The present application provides a data acquisition method, system, electronic device, storage medium and program product, which are used to solve the problem of incomplete image data of the free-form surface of a material acquired by a data acquisition system in the prior art.

[0005] In a first aspect, the present application provides a data collection method, comprising:

[0006] Acquire first sparse point cloud data of the free-form surface of the material;

[0007] Determining a moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor;

[0008] The material is moved based on the moving path, and the first line sensor is controlled to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0009] Optionally, determining the moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor includes:

[0010] Performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface;

[0011] A moving path of the material is determined based on the dense point cloud data and a sensing area of ​​the first line sensor.

[0012] Optionally, performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface includes:

[0013] Acquire the edge contour of the free-form surface according to the first sparse point cloud data;

[0014] Based on the first sparse point cloud data and the edge contour, a surface fitting is performed to obtain a surface function expression of the free-form surface;

[0015] The first sparse point cloud data is filled based on the surface function expression to obtain the dense point cloud data.

[0016] Optionally, acquiring the edge contour of the free-form surface according to the first sparse point cloud data includes:

[0017] Preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data;

[0018] Based on the boundary point cloud of the preprocessed first sparse point cloud data, an edge contour of the free-form surface is fitted.

[0019] Optionally, the preprocessing includes noise reduction processing, and the preprocessing of the first sparse point cloud data to obtain the preprocessed first sparse point cloud data includes:

[0020] The first sparse point cloud data is sequentially subjected to through filtering and statistical filtering to obtain filtered first sparse point cloud data.

[0021] Optionally, the preprocessing includes data completion processing, and the preprocessing of the first sparse point cloud data to obtain the preprocessed first sparse point cloud data includes:

[0022] Extracting boundary point cloud data from second sparse point cloud data, where the second sparse point cloud data is data collected by a second line sensor disposed on a side of the free-form surface;

[0023] The first sparse point cloud data is supplemented by using the boundary point cloud data, wherein the first sparse point cloud data is data collected by a point sensor disposed above the free-form surface.

[0024] Optionally, determining the moving path of the material based on the dense point cloud data and the sensing area of ​​the first line sensor includes:

[0025] Converting the dense point cloud data into first image data with depth information;

[0026] A moving path of the material is determined according to the first image data and a sensing area of ​​the first line sensor.

[0027] Optionally, determining the moving path of the material according to the first image data and the sensing area of ​​the first line sensor includes:

[0028] Determining an initial moving path of the material at a default height according to the first image data and a field of view width corresponding to the sensing area;

[0029] According to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area, the height of the initial moving path is adjusted to obtain the moving path.

[0030] Optionally, the initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface;

[0031] The adjusting the height of the initial moving path according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path includes:

[0032] For each scanning column, acquiring a height distribution of the scanning column based on depth information of the scanning column in the first image data;

[0033] Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path.

[0034] Optionally, adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes:

[0035] If the height distributions of the scanning columns are all located within the scanning range, determining a first height based on the height distributions of the scanning columns;

[0036] The height of the initial moving path is adjusted using the first height to obtain the moving path.

[0037] Optionally, adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes:

[0038] If the height distribution exceeds the scanning range, the height of the straight line path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

[0039] Optionally, adjusting the height of the straight line path based on the height of each row of the scanning column and the scanning range corresponding to the sensing area includes:

[0040] Calculate the height distribution from the starting row to the current row of the scan column. If the height distribution of the current row exceeds the scan range, use the current row as a discontinuity point, obtain the second height of the straight line path segment, and use the second height to adjust the height of the straight line path segment.

[0041] The current row is used as a new starting row to obtain the next breakpoint until the last row of the scan column is traversed.

[0042] Optionally, before determining the moving path of the material according to the first image data and the sensing area of ​​the first line sensor, the method further includes:

[0043] If the coordinate system of the first image data is different from the coordinate system where the first line sensor is located, coordinate transformation is performed on the first image data based on the coordinate system where the first line sensor is located.

[0044] Optionally, it also includes:

[0045] The scanned image data of the free-form surface are spliced ​​to obtain image data of the free-form surface.

[0046] Optionally, the step of stitching the scanned image data of the free-form surface to obtain the image data of the free-form surface includes:

[0047] splicing the scanned image data based on the path start and end points corresponding to the scanned image data to obtain an initial spliced ​​image;

[0048] The image data is obtained by fusing the overlapping parts of the adjacent scanned image data in the initial spliced ​​image.

[0049] Optionally, the data fusion processing of the overlapping parts of the adjacent scanned image data in the initial stitched image to obtain the image data includes:

[0050] Convert the data of the overlapping part into point cloud data; perform local registration on the point cloud data to obtain a transformation matrix of the data of the overlapping part;

[0051] Based on the transformation matrix, the data of the overlapping parts are fused to obtain the image data.

[0052] In a second aspect, the present application provides a data acquisition system, comprising: a first scanning device, a second scanning device, and a data processing device;

[0053] The first scanning device is used to collect first sparse point cloud data of the free-form surface of the material;

[0054] The second scanning device includes a first line sensor and a first moving device, wherein the first line sensor is arranged above the first moving device; the first moving device is used to carry materials;

[0055] The data processing device is used to determine the moving path of the material as the height of the free-form surface changes based on the first sparse point cloud data, using the method described in the first aspect and various possible designs of the first aspect, and control the first moving device to move the material based on the moving path, and control the first line sensor to scan the free-form surface to obtain scanning image data of the free-form surface at different heights.

[0056] Optionally, the first scanning device includes a point sensor and a second moving device, and the point sensor is arranged above the second moving device;

[0057] The point sensor is used to scan the free-form surface and collect first sparse point cloud data of the free-form surface of the material;

[0058] The second moving device is used to carry and move the material.

[0059] Optionally, the system further comprises: a third scanning device, the third scanning device comprising a second line sensor and a third moving device; the second line sensor is arranged on a side of the third moving device;

[0060] The second line sensor is used to scan the free-form surface and collect second sparse point cloud data of the free-form surface of the material;

[0061] The third moving device is used to carry and move the material.

[0062] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0063] The memory stores computer-executable instructions;

[0064] The processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect and various possible designs of the first aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect are implemented.

[0066] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect and various possible designs of the first aspect.

[0067] The data acquisition method, system, electronic device, storage medium and program product provided by the present application obtain the first sparse point cloud data of the free-form surface of the material, determine the moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor, move the material based on the moving path, and control the line sensor to scan the free-form surface to obtain scanning image data of different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the moving path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of ​​the line sensor, thereby scanning image data of different heights of the free-form surface, and obtaining complete image data of the free-form surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] Figure 1 A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 1 ;

[0070] Figure 2 A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 2 ;

[0071] Figure 3 A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 3 ;

[0072] Figure 4 A structural diagram of a data acquisition system provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the structure of a third scanning device provided in an embodiment of the present application;

[0074] Figure 6 A flow chart of a data collection method provided in an embodiment of the present application;

[0075] Figure 7 A schematic diagram of a first sparse point cloud data provided in an embodiment of the present application;

[0076] Figure 8 A schematic flow chart of a method for generating dense point cloud data of a free-form surface provided in an embodiment of the present application;

[0077] Fig. 9 A schematic diagram of a process for determining a moving path of a material provided in an embodiment of the present application;

[0078] Fig.10 A schematic diagram of a scanning column of an initial moving path provided in an embodiment of the present application;

[0079] Fig.11 A schematic diagram of a moving path provided in an embodiment of the present application Figure 1 ;

[0080] Fig.12 A schematic diagram of a process for determining a second height provided in an embodiment of the present application;

[0081] Fig.13 A schematic diagram of a moving path provided in an embodiment of the present application Figure 2 ;

[0082] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0083] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0085] Combine the following Figure 1 , Figure 2 and Figure 3 The data collection method in the related art is explained. Figure 1 A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 1 . Figure 2 A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 2 . Figure 3A schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 3 .

[0086] Reference Figure 1 As shown in the figure, the scanning range that the line sensor can scan is h, that is, the height of the free-form surface that can be scanned is h. Figure 2 As shown in the figure, the scanning line length that the line sensor can scan is l, that is, the width of the free-form surface that can be scanned is l. Figure 3 As shown, the scanning range h and Figure 2 The scanning line length is l, and the sensing area of ​​the line sensor is a rectangular area, and the length and width of the rectangular area are l and h respectively. Figure 3 As shown, the material is scanned with the y-axis direction as the scanning direction. When the height change of the free-form surface of the material exceeds the sensing area of ​​the line sensor, the line sensor cannot scan the portion of the free-form surface that exceeds the sensing area of ​​the line sensor. The height indicated by δ in the figure is the height of the free-form surface exceeding h. The line sensor cannot scan the portion of the free-form surface within the height indicated by δ, resulting in incomplete image data of the collected free-form surface.

[0087] In view of this, the present application proposes a data acquisition method. In the process of a line sensor scanning the free-form surface of a material, the material is controlled to move along with the height change of the free-form surface based on a moving path, so that the free-form surface scanned by the line sensor is within the sensing area of ​​the line sensor, thereby scanning image data of different heights of the free-form surface and obtaining complete image data of the free-form surface.

[0088] Based on the implementation of the data collection method provided in the embodiment of the present application, the present application proposes a data collection system. Figure 4 The structure of this data acquisition system will be described.

[0089] Figure 4 This is a schematic diagram of the structure of a data acquisition system provided in an embodiment of the present application. Figure 4 As shown, the data acquisition system includes: a first scanning device 401 , a second scanning device 402 and a data processing device 403 , and the second scanning device 402 includes a first line sensor 404 and a first moving device 405 .

[0090] The first scanning device 401 is used to collect first sparse point cloud data of the free-form surface of the material. The first scanning device 401 may include a scanning device such as a structured light scanner, an optical sensor, or a millimeter wave radar for scanning the free-form surface of the material. The free-form surface of the material can be scanned by the first scanning device 401 to obtain first sparse point cloud data of the free-form surface. The first sparse point cloud data is point cloud data collected with a low point density.

[0091] The first line sensor 404 is disposed above the first moving device 405. The first moving device 405 is used to carry the material and move the material based on the moving path determined by the data processing device 403, and the moving path is the moving path of the material as the height of the free-form surface changes. The first line sensor 404 is used to scan the free-form surface and the scanned image data of the free-form surface of the material. The first moving device 405 can drive the material to move in the horizontal direction and the vertical direction.

[0092] The data processing device 403 is used to determine the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the data acquisition method provided in the embodiment of the present application, and control the first moving device 405 to move the material based on the movement path, and control the first line sensor 404 to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0093] The data processing device 403 may be, for example, any device capable of performing data processing, and the device is not limited to a specific implementation method, and may be implemented by software and / or hardware. Specifically, the data processing device may be, for example, an electronic device or a processor.

[0094] Reference Figure 1 The first scanning device 401 shown in the figure exemplarily includes a point sensor 406 and a second moving device 407 , and the point sensor 406 is arranged above the second moving device 407 .

[0095] The point sensor 406 is used to scan the free-form surface and collect the first sparse point cloud data of the free-form surface of the material. The second moving device 407 is used to carry and move the material.

[0096] For example, in the process of horizontally moving the material by the second moving device 407, the free-form surface of the material is scanned by the point sensor 406. The point sensor 406 can collect point cloud data of multiple points on the free-form surface of the material by scanning point by point, that is, the first sparse point cloud data.

[0097] Optionally, the data acquisition system may further include: a third scanning device.

[0098] Figure 5 Schematic diagram of the structure of a third scanning device provided in an embodiment of the present application. Figure 5 As shown, the third scanning device includes a second line sensor 501 and a third moving device 502 , and the second line sensor 501 is arranged on the side of the third moving device 502 .

[0099] The second line sensor 501 is used to scan the free-form surface and collect the second sparse point cloud data of the free-form surface of the material. The third moving device 502 is used to carry and move the material.

[0100] For example, when a price material is placed on the third moving device 502, the free-form surface of the material is placed upward, and the third moving device 502 in the third scanning device can rotate and move the material, and scan the side of the material through the second line sensor 501. Furthermore, second sparse point cloud data can be obtained from the image data of the side collected by the second line sensor, and the second sparse point cloud data is point cloud data of the free-form surface contour.

[0101] Optionally, the data acquisition system further includes: a displacement device.

[0102] The displacement device can be used to receive displacement instructions sent by the data processing device 403 to move the material from the second displacement device to the third displacement device, and move the material from the second displacement device 407 to the third displacement device 502. It can also receive displacement instructions sent by the data processing device 403 to move the material from the third displacement device to the first displacement device, and move the material from the third displacement device 502 to the first displacement device 405.

[0103] The data acquisition method provided by the embodiment of the present application is described in detail below by taking the above-mentioned data acquisition system as an example. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0104] It should be understood that the data collection method of the embodiment of the present application can be used in any scenario of collecting image data of a free-form surface of a material. The following description is made by taking the execution subject as the above-mentioned data processing device 403 as an example.

[0105] Figure 6 A flow chart of a data collection method provided in an embodiment of the present application. Figure 6 As shown in , the data collection method may include the following steps:

[0106] S601. Acquire first sparse point cloud data of a free-form surface of a material.

[0107] The first sparse point cloud data is relatively scattered point cloud data on the free-form surface of the material, and the first sparse point cloud data is data represented based on three-dimensional coordinates.

[0108] Exemplarily, the data processing device 403 may obtain first sparse point cloud data obtained by scanning the free-form surface of the material by the point sensor 406 in the first scanning device 401. For example, the data processing device 403 may control the first scanning device 401 to scan the free-form surface of the material, and the first scanning device 401 obtains a plurality of point cloud data of the free-form surface of the material by scanning through the point sensor 406 and the second moving device 407.

[0109] Figure 7 A schematic diagram of a first sparse point cloud data provided in an embodiment of the present application. Figure 7 , which is a schematic projection diagram of first sparse point cloud data collected on a free-form surface, wherein each scanning point in the figure is point cloud data scanned and acquired by the first scanning device 401 .

[0110] S602: Determine a moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor.

[0111] The sensing area of ​​the first line sensor 404 is determined by its hardware parameters. The hardware parameters of the first line sensor 404 include a scanning line length and a scanning range. The scanning range defines the depth range scanned by the first line sensor 404 , and the scanning line length defines the width scanned by the first line sensor 404 .

[0112] The moving path is the moving path of the material during the scanning process of the first line sensor 404. The moving path may include movement in the horizontal direction and the vertical direction.

[0113] Exemplarily, the data processing device 403 can convert the first sparse point cloud data into image data with depth information, where the depth information is information determined by the vertical coordinate axis of each point cloud data in the first sparse point cloud data in the three-dimensional coordinates. Based on the image data obtained from the first sparse point cloud data, the moving path of the material as the height of the free surface changes can be determined.

[0114] For example, when the height of the material is at the default height, if it is determined based on the depth information that a portion of the free-form surface is within the scanning area that can be scanned by the first line sensor 404, then it can be determined that when the first line sensor 404 scans the area, the height of the material movement is the default height; if it is determined based on the depth information that a portion of the free-form surface is lower in height than the scanning area that can be scanned by the first line sensor 404, then it can be determined that when the first line sensor 404 scans the area, the height of the material movement is a height adjusted downward from the default height.

[0115] Exemplarily, the data processing device 403 may use the mean of the depth information of the corresponding image data in the area and the difference of the default height as the height of the material movement, or may use the mean of the maximum depth information and the minimum depth information of the depth information of the corresponding image data in the area and the difference of the default height as the height of the material movement.

[0116] S603, moving the material based on the moving path, and controlling the first line sensor to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0117] The scanned image data is image data obtained by scanning through the first line sensor 404 .

[0118] Exemplarily, the data processing device 403 can obtain scanning image data at different heights of the free-form surface through the second scanning device 402. The data processing device 403 can control the first moving device 405 to move the material based on the moving path, and control the first line sensor 404 to scan the free-form surface, and obtain the image data scanned by the first line sensor 404. The data processing device 403 can obtain scanning image data at different heights of the free-form surface based on the adjustment of the material in height.

[0119] The data acquisition method of the present application obtains the first sparse point cloud data of the free-form surface of the material, determines the moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor, moves the material based on the moving path, and controls the line sensor to scan the free-form surface to obtain scanning image data of different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the moving path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of ​​the line sensor, thereby scanning image data of different heights of the free-form surface to obtain complete image data of the free-form surface.

[0120] The following describes a method for determining the moving path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of ​​the line sensor.

[0121] Optionally, the data processing device may perform point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface, and determine the movement path of the material based on the dense point cloud data and the sensing area of ​​the first line sensor.

[0122] The dense point cloud data has a larger data volume and a smaller point spacing than the first sparse point cloud data.

[0123] Figure 8A flowchart of a method for generating dense point cloud data of a free-form surface provided in an embodiment of the present application is provided. For example, Figure 8 As shown, performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface may include the following steps:

[0124] S801. Acquire an edge contour of a free-form surface according to first sparse point cloud data.

[0125] Exemplarily, the data processing device 403 can extract the boundary point cloud of the first sparse point cloud data based on the edge detection method in the related art, connect the detected boundary point cloud into an initial edge contour, smooth the initial edge contour, and obtain the edge contour of the free surface.

[0126] Exemplarily, the data processing device 403 may preprocess the first sparse point cloud data to obtain preprocessed first sparse point cloud data, and fit the edge contour of the free-form surface based on the boundary point cloud of the preprocessed first sparse point cloud data.

[0127] In a possible implementation, preprocessing includes noise reduction processing. The data processing device 403 preprocesses the first sparse point cloud data to obtain preprocessed first sparse point cloud data. The first sparse point cloud data can be filtered by sequentially performing through filtering and statistical filtering on the first sparse point cloud data to obtain filtered first sparse point cloud data.

[0128] Through filtering is a filtering method that removes points in point cloud data that exceed a specified range. The retained range is limited by setting the maximum and minimum values, as shown in formula (1):

[0129] Formula (1)

[0130] in, is any point cloud in the first sparse point cloud data, is the maximum value set. is the minimum value set. The point cloud that satisfies formula (1) is a point cloud that exceeds the specified range. The point cloud that exceeds the specified range is deleted from the first sparse point cloud data.

[0131] Statistical filtering, as shown in formula (2):

[0132] Formula (2)

[0133] in, is the average distance from any point cloud in the first sparse point cloud data to other point clouds in its neighborhood, is the mean of the average distances of all point clouds in the neighborhood, is the standard deviation of the average distance of all point clouds in the neighborhood, is a threshold coefficient, which is used to determine the multiple of the standard deviation. The point cloud that satisfies formula (2) is a noise point, and the noise point is deleted from the first sparse point cloud data.

[0134] A possible implementation method is that the preprocessing includes data completion processing. The data processing device 403 preprocesses the first sparse point cloud data to obtain the preprocessed first sparse point cloud data. The boundary point cloud data can be extracted from the second sparse point cloud data. The second sparse point cloud data is the data collected by the second line sensor 501 set on the side of the free-form surface. The boundary point cloud data is used to supplement the first sparse point cloud data. The first sparse point cloud data is the data collected by the point sensor set above the free-form surface.

[0135] For example, the data processing device 403 rotates the material for one circle based on the third moving device 502, and controls the second line sensor 501 to scan and obtain image data of the side of the material. The boundary point cloud data of the contour of one side of the free-form surface is extracted from the acquired side image data, that is, the second sparse point cloud data. The first sparse point cloud data is supplemented based on the second sparse point cloud data to obtain the pre-processed first sparse point cloud data.

[0136] Based on the preprocessed first sparse point cloud data, a boundary point cloud is extracted, and the boundary point cloud is fitted into a curve, which is the edge contour of the free-form surface.

[0137] In this way, by preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data, the quality of the first sparse point cloud can be improved so that the edge contour of the free-form surface can be fitted more accurately based on the boundary point cloud based on the preprocessed first sparse point cloud data.

[0138] S802: Perform surface fitting based on the first sparse point cloud data and the edge contour to obtain a surface function expression of the free-form surface.

[0139] Exemplarily, the data processing device 403 may input the first sparse point cloud data and the edge contour into the surface fitting model through the surface fitting model in the related art to obtain the fitted surface function expression.

[0140] Exemplarily, the data processing device 403 can obtain the surface function expression of the free-form surface through the surface fitting algorithm in the related art. Taking the non-uniform rational B-splines (NURBS) algorithm as an example, as shown in formula (3):

[0141] Formula (3)

[0142] in, is a point on the NURBS surface, u and v are parameters in the parameter space, n and are the number of control points of the control mesh in the u direction and v direction respectively, p and q are the orders of the B-spline basis functions, and is the B-spline basis function defined on the knot vector, It is a control point The weight of are the control points on the control grid, and the edge contours define the range of the control points.

[0143] The NURBS algorithm adjusts the control points and weights by minimizing the error function to make the NURBS surface as close to the point cloud as possible. For example, the NURBS algorithm optimizes and minimizes the error function based on the least squares method. The error function can be the sum of the squares of the distance from the point to the surface, as shown in formula (4):

[0144] Formula (4)

[0145] in, It is the point corresponding to the point cloud Qk in the first sparse point cloud data in the NURBS surface.

[0146] S803 . Fill the first sparse point cloud data based on the surface function expression to obtain dense point cloud data.

[0147] Based on the surface function expression, multiple point cloud data can be determined, and the obtained multiple point cloud data are filled into the first sparse point cloud data to obtain dense point cloud data. For example, new point cloud data can be inserted between any adjacent point clouds in the first sparse point cloud data based on the interpolation method, and the value of the new point cloud data is obtained based on the surface function expression.

[0148] The data processing method of the embodiment of the present application can increase the data volume of the first sparse point cloud data by performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free surface, and provide data support for determining the movement path of the material based on the dense point cloud data and the sensing area of ​​the first-line sensor, thereby making the determined movement path more accurate.

[0149] Exemplarily, based on dense point cloud data and the sensing area of ​​the line sensor, the moving path of the material is determined. The data processing device 403 can convert the dense point cloud data into first image data with depth information, and determine the moving path of the material based on the first image data and the sensing area of ​​the line sensor.

[0150] Exemplarily, the data processing device 403 may convert the dense point cloud data into image data by projection, and retain the values ​​of the vertical coordinate axes of the dense point cloud data as depth information.

[0151] Optionally, if the coordinate system of the first image data is different from the coordinate system where the first line sensor is located, the coordinate transformation is performed on the first image data based on the coordinate system where the first line sensor is located.

[0152] For example, the coordinate system of the first image data is data based on the coordinate system of the point sensor 406. When the scanning range of the point sensor 406 is x and the scanning range of the first line sensor 404 is y, the depth information of the first image data can be increased by the difference of yx.

[0153] Fig. 9 A schematic diagram of a process for determining a moving path of a material provided in an embodiment of the present application. Fig. 9 As shown, determining the moving path of the material according to the first image data and the sensing area of ​​the line sensor may include the following steps:

[0154] S901. Determine an initial moving path of the material at a default height according to the first image data and the field of view width corresponding to the sensing area.

[0155] The default height is the preset material height, refer to Figure 4 For the first moving device 405 in the second scanning device 402 shown, the default height of the material can be regarded as the default height of the first moving device 405 .

[0156] The initial moving path is the moving path of the material in the horizontal direction.

[0157] Exemplarily, the data processing device 403 can determine the initial moving path of the material at a default height according to the bow-shaped path based on the image data and the field of view width corresponding to the sensing area as the path width, and the initial moving path covers the first image data.

[0158] Exemplarily, the data processing device 403 can use the field of view width corresponding to the sensing area of ​​the image data as the path width, starting from the image data at the leftmost boundary of the first image data, from top to bottom as the moving path of the first column, and moving right to the image data at the bottom boundary of the second column, from bottom to top as the moving path of the second column, and moving right to the image data at the top boundary of the third column, according to this planning method, until the first image data is covered, the moving paths of each column and the moving path between two adjacent columns are connected to determine as the initial moving path.

[0159] Optionally, adjacent scanning columns may have overlapping areas. Taking the initial moving path as a bow-shaped path as an example, for example, adjacent straight line segment paths may overlap by 20% of their area. Fig.10 A schematic diagram of a scanning column of a starting moving path provided in an embodiment of the present application. Fig.10As shown, the moving paths in each scanning column are indicated by arrows, and adjacent columns may partially overlap.

[0160] S902: According to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area, the height of the initial moving path is adjusted to obtain the moving path.

[0161] Exemplarily, the initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface. Fig.11 A schematic diagram of a moving path provided in an embodiment of the present application Figure 1 .like Fig.11 As shown, the left boundary of the rectangular frame is a straight line segment path, and the area shown in the rectangular frame is a scanning column corresponding to the straight line segment path. The width of the scanning column is determined by the scanning line length of the first line sensor 404.

[0162] The data processing device 403 can obtain the height distribution of each scanning column based on the depth information of the scanning column in the first image data, and adjust the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path.

[0163] Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path. A possible implementation method is that if the height distribution of the scanning column is located within the scanning range, a first height is determined based on the height distribution of the scanning column, and the height of the initial moving path is adjusted using the first height to obtain the moving path.

[0164] Reference Fig.11 As shown, for example, the data processing device 403 can read the depth information of the first image data in the scan column, obtain the height distribution of the scan column based on the data distribution algorithm, and if the height of the first image data of the scan column is within the scan range, the average value of the height of the first image data of the scan column is used as the first height, and the height of the initial moving path is adjusted using the first height. For example, if the default height is 10 mm and the first height is 15 mm, the material can be moved 5 mm in the vertical direction when moving based on the default height.

[0165] In a possible implementation, if the height distribution exceeds the scanning range, the height of the straight line segment path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

[0166] Exemplarily, the data processing device 403 may calculate the height distribution from the starting row to the current row of the scan column. If the height distribution of the current row exceeds the scan range, the current row is used as a discontinuity point to obtain the second height of the straight line path segment, and the height of the straight line path segment is adjusted using the second height. The current row is used as a new starting row to obtain the next discontinuity point until the last row of the scan column is traversed.

[0167] Fig.12 A schematic diagram of a process for determining a second height provided in an embodiment of the present application. Fig.12 As shown, for any scan column, it specifically includes:

[0168] S1201. Set the current row as the starting row.

[0169] S1202: Determine whether the current row height distribution exceeds the scanning range.

[0170] If yes, execute S1204;

[0171] If not, execute S1203.

[0172] S1203: Obtain height distribution of depth information of the first image data of the next row.

[0173] After obtaining the height distribution of the depth information of the first image data of the next row, the process continues with S1202 .

[0174] S1204. Take the current line as a discontinuity point and obtain the second height of the straight line path segment.

[0175] The straight line path segment is from the starting line to the line before the current line.

[0176] Exemplarily, the data processing device 403 may use the average of the depth information of the first image data from the starting row of the scan column to the previous row of the current row as the second height of the straight line path segment. Optionally, the data processing device 403 may also determine the maximum and minimum values ​​of the depth information of the first image data from the starting row of the scan column to the previous row of the current row, and use the average of the maximum and minimum values ​​as the second height of the straight line path segment.

[0177] The current row is used as a discontinuity point, and after obtaining the second height of the straight line path segment, S1201 is continued to be executed until the current row becomes the last row of the scan column, and then other scan columns are processed column by column.

[0178] Based on the first height or the second height obtained in the above embodiment, the height of the initial moving path is adjusted using the first height or the second height to obtain the moving path. Fig.13 A schematic diagram of a moving path provided in an embodiment of the present application Figure 2 .like Fig.13As shown, each point is the position of a row of the scanning column, each column is a straight line path, the first column on the far right includes two points, that is, the moving path of the first column has not changed in height, the fourth column includes four points, that is, the straight line path segment from the first point to the second point has a height, the straight line path segment from the second point to the third point has a height, and the straight line path segment from the third point to the fourth point has a height, that is, in the process of scanning the free curve surface where the straight line path is located, the height of the material needs to be adjusted once at the second point and the third point respectively.

[0179] The data processing device 403 moves the material based on the moving path, and controls the first line sensor 404 to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0180] The data acquisition method of the present application obtains the first sparse point cloud data of the free-form surface of the material, determines the moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor, moves the material based on the moving path, and controls the line sensor to scan the free-form surface to obtain scanning image data of different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the moving path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of ​​the line sensor, thereby scanning image data of different heights of the free-form surface to obtain complete image data of the free-form surface.

[0181] Optionally, the data processing device 403 may also splice the scanned image data of the free-form surface based on the obtained scanned image data of the free-form surface at different heights to obtain the image data of the free-form surface.

[0182] Exemplarily, the data processing device 403 may stitch the scanned image data based on the path start and end points corresponding to the scanned image data to obtain an initial stitched image, and fuse the data of the overlapping parts of adjacent scanned image data in the initial stitched image to obtain image data.

[0183] Based on the starting and ending points of the path corresponding to the scanned image data, the scanned image data is spliced ​​to obtain an initial spliced ​​image. The data processing device 403 can determine the splicing position of each scanned image according to the starting and ending points of the path corresponding to the scanned image data, and splice each scanned image based on the image splicing method according to the splicing position of each scanned image to obtain an initial spliced ​​image.

[0184] In this way, the scanned image data can be quickly stitched together to obtain the initial free-form surface image data.

[0185] The data of the overlapping parts in the adjacent scanned image data in the initial stitched image are fused to obtain image data. The data processing device 403 can convert the data of the overlapping parts into point cloud data, perform local alignment on the point cloud data, and obtain the transformation matrix of the data of the overlapping parts. Based on the transformation matrix, the data of the overlapping parts are fused to obtain image data.

[0186] Specifically, the two sets of overlapping data are converted into point cloud data, local alignment is performed based on the feature matching method, the transformation matrix of the two sets of point cloud data is calculated, one set of point cloud data is transformed according to the transformation matrix, the converted point cloud data is aligned with the other set of point cloud data, and fusion processing is performed to complete the precise stitching of the image containing the overlapping data and obtain the image data of the free-form surface of the material.

[0187] In the method of the embodiment of the present application, the scanned image data are spliced ​​based on the starting and ending points of the path corresponding to the scanned image data to obtain an initial spliced ​​image, and the data of the overlapping parts of adjacent scanned image data in the initial spliced ​​image are fused to obtain image data, which is image data of the free-form surface of the material after more accurate splicing.

[0188] The above is an explanation of the method embodiment of the present application. The data processing device 403 provided in the embodiment of the present application can execute the above method embodiment.

[0189] Fig.14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.14 As shown, the electronic device may include: at least one processor 1401 and a memory 1402 .

[0190] The memory 1402 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0191] The memory 1402 may include a high-speed RAM memory, and may also include a non-volatile memory.

[0192] The processor 1401 is used to execute the computer-executable instructions stored in the memory 1402 to implement the method of the aforementioned method embodiment. The processor 1401 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0193] Optionally, the electronic device may further include a communication interface 1403. In a specific implementation, if the communication interface 1403, the memory 1402 and the processor 1401 are implemented independently, the communication interface 1403, the memory 1402 and the processor 1401 may be interconnected via a bus and communicate with each other.

[0194] Optionally, in a specific implementation, if the communication interface 1403, the memory 1402 and the processor 1401 are integrated on a chip, the communication interface 1403, the memory 1402 and the processor 1401 can communicate through an internal interface.

[0195] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used to implement the actions of the above-mentioned method implementation method.

[0196] The present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the actions of the above method implementation are implemented.

[0197] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0198] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0199] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk and other media that can store program codes.

[0200] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in any computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM and other media that can store program codes.

Claims

1. A data collection method, characterized in that: include: Acquire first sparse point cloud data of the free-form surface of the material; Determining a moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor; The material is moved based on the moving path, and the first line sensor is controlled to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

2. The method according to claim 1, characterized in that The step of determining the moving path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of ​​the first line sensor comprises: Performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface; A moving path of the material is determined based on the dense point cloud data and a sensing area of ​​the first line sensor.

3. The method according to claim 2, characterized in that The step of performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface includes: Acquire the edge contour of the free-form surface according to the first sparse point cloud data; Based on the first sparse point cloud data and the edge contour, a surface fitting is performed to obtain a surface function expression of the free-form surface; The first sparse point cloud data is filled based on the surface function expression to obtain the dense point cloud data.

4. The method according to claim 3, characterized in that The step of acquiring the edge contour of the free-form surface according to the first sparse point cloud data comprises: Preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data; Based on the boundary point cloud of the preprocessed first sparse point cloud data, an edge contour of the free-form surface is fitted.

5. The method according to claim 4, characterized in that The preprocessing includes noise reduction processing, and the preprocessing of the first sparse point cloud data to obtain the preprocessed first sparse point cloud data includes: The first sparse point cloud data is sequentially subjected to through filtering and statistical filtering to obtain filtered first sparse point cloud data.

6. The method according to claim 4, characterized in that The preprocessing includes data completion processing, and the preprocessing of the first sparse point cloud data to obtain the preprocessed first sparse point cloud data includes: Extracting boundary point cloud data from second sparse point cloud data, where the second sparse point cloud data is data collected by a second line sensor disposed on a side of the free-form surface; The first sparse point cloud data is supplemented by using the boundary point cloud data, wherein the first sparse point cloud data is data collected by a point sensor disposed above the free-form surface.

7. The method according to any one of claims 2 to 6, characterized in that: The determining the moving path of the material based on the dense point cloud data and the sensing area of ​​the first line sensor includes: Converting the dense point cloud data into first image data with depth information; A moving path of the material is determined according to the first image data and a sensing area of ​​the first line sensor.

8. The method according to claim 7, characterized in that The step of determining the moving path of the material according to the first image data and the sensing area of ​​the first line sensor includes: Determining an initial moving path of the material at a default height according to the first image data and a field of view width corresponding to the sensing area; According to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area, the height of the initial moving path is adjusted to obtain the moving path.

9. The method according to claim 8, characterized in that The initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface; The adjusting the height of the initial moving path according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path includes: For each scanning column, acquiring a height distribution of the scanning column based on depth information of the scanning column in the first image data; Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path.

10. The method according to claim 9, characterized in that The step of adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path comprises: If the height distributions of the scanning columns are all located within the scanning range, determining a first height based on the height distributions of the scanning columns; The height of the initial moving path is adjusted using the first height to obtain the moving path.

11. The method according to claim 9, characterized in that The step of adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path comprises: If the height distribution exceeds the scanning range, the height of the straight line path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

12. The method according to claim 11, characterized in that The adjusting the height of the straight line path based on the height of each row of the scanning column and the scanning range corresponding to the sensing area includes: Calculate the height distribution from the starting row to the current row of the scan column. If the height distribution of the current row exceeds the scan range, use the current row as a discontinuity point, obtain the second height of the straight line path segment, and use the second height to adjust the height of the straight line path segment. The current row is used as a new starting row to obtain the next breakpoint until the last row of the scan column is traversed.

13. The method according to claim 7, characterized in that Before determining the moving path of the material according to the first image data and the sensing area of ​​the first line sensor, the method further includes: If the coordinate system of the first image data is different from the coordinate system where the first line sensor is located, coordinate transformation is performed on the first image data based on the coordinate system where the first line sensor is located.

14. The method according to any one of claims 1 to 6, characterized in that: Also includes: The scanned image data of the free-form surface are spliced ​​to obtain image data of the free-form surface.

15. The method according to claim 14, characterized in that The step of stitching the scanned image data of the free-form surface to obtain the image data of the free-form surface includes: splicing the scanned image data based on the path start and end points corresponding to the scanned image data to obtain an initial spliced ​​image; The image data is obtained by fusing the overlapping parts of the adjacent scanned image data in the initial spliced ​​image.

16. The method according to claim 15, characterized in that The data fusion processing of the overlapping parts of the adjacent scanned image data in the initial spliced ​​image to obtain the image data includes: Convert the data of the overlapping part into point cloud data; perform local registration on the point cloud data to obtain a transformation matrix of the data of the overlapping part; Based on the transformation matrix, the data of the overlapping parts are fused to obtain the image data.

17. A data acquisition system, characterized in that: include: A first scanning device, a second scanning device and a data processing device; The first scanning device is used to collect first sparse point cloud data of the free-form surface of the material; The second scanning device includes a first line sensor and a first moving device, wherein the first line sensor is arranged above the first moving device; the first moving device is used to carry materials; The data processing device is used to determine the moving path of the material as the height of the free-form surface changes based on the first sparse point cloud data and using the method described in any one of claims 1 to 16, and control the first moving device to move the material based on the moving path, and control the first line sensor to scan the free-form surface to obtain scanning image data of the free-form surface at different heights.

18. The system according to claim 17, characterized in that The first scanning device includes a point sensor and a second moving device, and the point sensor is arranged above the second moving device; The point sensor is used to scan the free-form surface and collect first sparse point cloud data of the free-form surface of the material; The second moving device is used to carry and move the material.

19. The system according to claim 17 or 18, characterized in that The system further comprises: a third scanning device, the third scanning device comprising a second line sensor and a third moving device; the second line sensor is arranged on a side of the third moving device; The second line sensor is used to scan the free-form surface and collect second sparse point cloud data of the free-form surface of the material; The third moving device is used to carry and move the material.

20. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 16.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 16.

22. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 16 when being executed by a processor.

Images