Data acquisition method, system, electronic device, storage medium and program product

Abstract

The present application provides a data acquisition method, system, electronic device, storage medium, and program product. The method includes: acquiring first sparse point cloud data of a free-form surface of a material; determining a movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of a first line sensor; moving the material based on the movement path, and controlling the first line sensor to scan the free-form surface to obtain scanned image data at different heights of the free-form surface. The method of the present application can scan image data at different heights of the free-form surface to obtain complete image data of the free-form surface.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning measurement technology, and in particular to a data acquisition method, system, electronic device, storage medium and program product. Background Art

[0002] Free-form surface data acquisition is used to collect image data on surfaces with irregular geometric features. In industrial material manufacturing, for materials with free-form surfaces, free-form surface data acquisition methods can be used to collect image data on the material surface for surface defect detection.

[0003] In related technologies, a data acquisition system controls the horizontal movement of a material. During this movement, a line sensor fixed above the material scans the material's free-form surface, generating image data of the free-form surface. However, due to the limited sensing area of the line sensor, if the height variation of the free-form surface exceeds the line sensor's sensing area, the line sensor cannot scan the portion of the free-form surface that exceeds the line sensor's sensing area, resulting in incomplete image data of the free-form surface. Summary of the Invention

[0004] The present application provides a data acquisition method, system, electronic device, storage medium and program product to solve the problem of incomplete image data of the free-form surface of a material acquired by a data acquisition system in the prior art.

[0005] In a first aspect, the present application provides a data collection method, comprising:

[0006] Acquire first sparse point cloud data of the free-form surface of the material;

[0007] determining a movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and a sensing area of the first line sensor;

[0008] The material is moved based on the moving path, and the first line sensor is controlled to scan the free-form surface to obtain scanning image data of the free-form surface at different heights.

[0009] Optionally, determining the movement path of the material as the height of the free-form surface changes according to the first sparse point cloud data and the sensing area of the first line sensor includes:

[0010] Performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface;

[0011] A moving path of the material is determined based on the dense point cloud data and a sensing area of the first line sensor.

[0012] Optionally, performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface includes:

[0013] Acquire an edge contour of the free-form surface according to the first sparse point cloud data;

[0014] Performing surface fitting based on the first sparse point cloud data and the edge contour to obtain a surface function expression of the free-form surface;

[0015] The first sparse point cloud data is filled based on the surface function expression to obtain the dense point cloud data.

[0016] Optionally, acquiring the edge contour of the free-form surface according to the first sparse point cloud data includes:

[0017] Preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data;

[0018] Based on the boundary point cloud of the preprocessed first sparse point cloud data, an edge contour of the free-form surface is fitted.

[0019] Optionally, the preprocessing includes noise reduction processing, and the preprocessing of the first sparse point cloud data to obtain preprocessed first sparse point cloud data includes:

[0020] The first sparse point cloud data is sequentially subjected to through-filtering and statistical filtering to obtain filtered first sparse point cloud data.

[0021] Optionally, the preprocessing includes data padding processing, and the preprocessing of the first sparse point cloud data to obtain preprocessed first sparse point cloud data includes:

[0022] Extracting boundary point cloud data from second sparse point cloud data, where the second sparse point cloud data is data collected by a second line sensor disposed on a side surface of the free-form surface;

[0023] The first sparse point cloud data is supplemented by using the boundary point cloud data, where the first sparse point cloud data is data collected by a point sensor disposed above the free-form surface.

[0024] Optionally, determining the moving path of the material based on the dense point cloud data and the sensing area of the first line sensor includes:

[0025] Converting the dense point cloud data into first image data with depth information;

[0026] A moving path of the material is determined according to the first image data and a sensing area of the first line sensor.

[0027] Optionally, determining the moving path of the material according to the first image data and the sensing area of the first line sensor includes:

[0028] determining an initial moving path of the material at a default height based on the first image data and a field of view width corresponding to the sensing area;

[0029] The height of the initial moving path is adjusted according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path.

[0030] Optionally, the initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface;

[0031] The step of adjusting the height of the initial moving path according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path includes:

[0032] For each scanning column, acquiring a height distribution of the scanning column based on depth information of the scanning column in the first image data;

[0033] Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path.

[0034] Optionally, adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes:

[0035] If the height distributions of the scanning rows are all located within the scanning range, determining a first height based on the height distributions of the scanning rows;

[0036] The height of the initial moving path is adjusted using the first height to obtain the moving path.

[0037] Optionally, adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes:

[0038] If the height distribution exceeds the scanning range, the height of the straight path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

[0039] Optionally, adjusting the height of the straight line path based on the height of each row of the scanning column and the scanning range corresponding to the sensing area includes:

[0040] Calculating the height distribution from the starting row to the current row of the scan column; if the height distribution of the current row exceeds the scan range, using the current row as a discontinuity point, obtaining a second height of the straight line path segment, and adjusting the height of the straight line path segment using the second height;

[0041] The current row is used as a new starting row to obtain the next breakpoint until the last row of the scan column is traversed.

[0042] Optionally, before determining the moving path of the material according to the first image data and the sensing area of the first line sensor, the method further includes:

[0043] If the coordinate system of the first image data is different from the coordinate system of the first line sensor, coordinate conversion is performed on the first image data based on the coordinate system of the first line sensor.

[0044] Optionally, it also includes:

[0045] The scanned image data of the free-form surface are spliced to obtain image data of the free-form surface.

[0046] Optionally, the step of stitching the scanned image data of the free-form surface to obtain the image data of the free-form surface includes:

[0047] splicing the scanned image data based on the path start and end points corresponding to the scanned image data to obtain an initial spliced image;

[0048] The image data is obtained by performing data fusion processing on overlapping parts of adjacent scanned image data in the initial spliced image.

[0049] Optionally, the fusing of overlapping portions of adjacent scanned image data in the initial stitched image to obtain the image data includes:

[0050] Converting the data of the overlapping part into point cloud data; performing local registration on the point cloud data to obtain a transformation matrix of the data of the overlapping part;

[0051] Based on the transformation matrix, the data of the overlapping parts are fused to obtain the image data.

[0052] In a second aspect, the present application provides a data acquisition system, comprising: a first scanning device, a second scanning device, and a data processing device;

[0053] The first scanning device is used to collect first sparse point cloud data of the free-form surface of the material;

[0054] The second scanning device includes a first line sensor and a first moving device, wherein the first line sensor is arranged above the first moving device; the first moving device is used to carry materials;

[0055] The data processing device is used to determine the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data, using the method described in the first aspect and various possible designs of the first aspect, and control the first moving device to move the material based on the movement path, and control the first line sensor to scan the free-form surface to obtain scanning image data of the free-form surface at different heights.

[0056] Optionally, the first scanning device includes a point sensor and a second moving device, and the point sensor is arranged above the second moving device;

[0057] The point sensor is used to scan the free-form surface and collect first sparse point cloud data of the free-form surface of the material;

[0058] The second moving device is used to carry and move the material.

[0059] Optionally, the system further comprises: a third scanning device, the third scanning device comprising a second line sensor and a third moving device; the second line sensor is arranged on a side of the third moving device;

[0060] The second line sensor is used to scan the free-form surface and collect second sparse point cloud data of the free-form surface of the material;

[0061] The third moving device is used to carry and move the material.

[0062] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0063] The memory stores computer-executable instructions;

[0064] The processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect and various possible designs of the first aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect is implemented.

[0066] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect and various possible designs of the first aspect.

[0067] The data acquisition method, system, electronic device, storage medium, and program product provided by the present application obtain first sparse point cloud data of the free-form surface of a material, determine the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of a first line sensor, move the material based on the movement path, and control the line sensor to scan the free-form surface to obtain scanned image data at different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the movement path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of the line sensor, thereby scanning image data at different heights of the free-form surface and obtaining complete image data of the free-form surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0069] Figure 1 Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 1 ;

[0070] Figure 2 Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 2 ;

[0071] Figure 3 Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 3 ;

[0072] Figure 4 A schematic diagram of the structure of a data acquisition system provided in an embodiment of the present application;

[0073] Figure 5 A schematic structural diagram of a third scanning device provided in an embodiment of the present application;

[0074] Figure 6 A flow chart of a data collection method provided in an embodiment of the present application;

[0075] Figure 7 A schematic diagram of first sparse point cloud data provided in an embodiment of the present application;

[0076] Figure 8 A schematic flow chart of a method for generating dense point cloud data of a free-form surface provided in an embodiment of the present application;

[0077] Figure 9 A schematic diagram of a process for determining a material's movement path provided in an embodiment of the present application;

[0078] Figure 10 A schematic diagram of a scanning column of an initial moving path provided in an embodiment of the present application;

[0079] Figure 11 A schematic diagram of a moving path provided in an embodiment of the present application Figure 1 ;

[0080] Figure 12 A schematic diagram of a process for determining a second height provided in an embodiment of the present application;

[0081] Figure 13 A schematic diagram of a moving path provided in an embodiment of the present application Figure 2 ;

[0082] Figure 14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0083] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0084] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0085] The following combination Figure 1 、 Figure 2 and Figure 3 The data collection method in the related art is described. Figure 1 Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 1 . Figure 2 Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 2 . Figure 3Schematic diagram of a line sensor scanning a free-form surface of a material provided in an embodiment of the present application Figure 3 .

[0086] Reference Figure 1 As shown in the figure, the scanning range that the line sensor can scan is h, that is, the height of the free-form surface that can be scanned is h. Figure 2 As shown in the figure, the line sensor can scan a line length of l, that is, it can scan a free-form surface with a width of l. Figure 3 As shown, the scanning range h and Figure 2 The scanning line length is l, and the sensing area of the line sensor is a rectangular area, the length and width of the rectangular area are l and h respectively. Figure 3 As shown, when the material is scanned in the y-axis direction, when the height change of the free-form surface of the material exceeds the sensing area of the line sensor, the line sensor cannot scan the portion of the free-form surface that exceeds the sensing area of the line sensor. The height shown by δ in the figure is the height of the free-form surface exceeding h, and the line sensor cannot scan the portion of the free-form surface within the height shown by δ, resulting in incomplete image data of the free-form surface.

[0087] In view of this, the present application proposes a data acquisition method. In the process of a line sensor scanning the free-form surface of a material, the material is controlled to move along with the height change of the free-form surface based on a moving path, so that the free-form surface scanned by the line sensor is within the sensing area of the line sensor, thereby scanning image data at different heights of the free-form surface and obtaining complete image data of the free-form surface.

[0088] Based on the implementation of the data acquisition method provided in the embodiment of the present application, the present application proposes a data acquisition system, which is combined with Figure 4 The structure of this data acquisition system will be described.

[0089] Figure 4 This is a structural diagram of a data acquisition system provided in an embodiment of the present application. Figure 4 As shown, the data acquisition system includes: a first scanning device 401 , a second scanning device 402 and a data processing device 403 , and the second scanning device 402 includes a first line sensor 404 and a first moving device 405 .

[0090] The first scanning device 401 is used to collect first sparse point cloud data of the free-form surface of the material. The first scanning device 401 may include a scanning device such as a structured light scanner, an optical sensor, or a millimeter-wave radar for scanning the free-form surface of the material. The first scanning device 401 can scan the free-form surface of the material to obtain first sparse point cloud data of the free-form surface. The first sparse point cloud data is collected with a low point density.

[0091] A first line sensor 404 is positioned above a first moving device 405. The first moving device 405 is used to carry the material and move it based on a movement path determined by the data processing device 403. The movement path is a path that varies with the height of the free-form surface. The first line sensor 404 is used to scan the free-form surface and generate scanned image data of the material's free-form surface. The first moving device 405 can move the material horizontally and vertically.

[0092] The data processing device 403 is used to determine the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the data acquisition method provided in the embodiment of the present application, and control the first moving device 405 to move the material based on the movement path, and control the first line sensor 404 to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0093] The data processing device 403 may be, for example, any device capable of processing data, and is not limited to a specific implementation method, and may be implemented by software and / or hardware. Specifically, the data processing device may be, for example, an electronic device or a processor.

[0094] Reference Figure 1 The first scanning device 401 shown exemplarily includes a point sensor 406 and a second moving device 407 , and the point sensor 406 is disposed above the second moving device 407 .

[0095] The point sensor 406 is used to scan the free-form surface and collect first sparse point cloud data of the free-form surface of the material. The second moving device 407 is used to carry and move the material.

[0096] For example, in the process of horizontally moving the material by the second moving device 407, the free curved surface of the material is scanned by the point sensor 406. The point sensor 406 can collect point cloud data of multiple points on the free curved surface of the material by scanning point by point, that is, the first sparse point cloud data.

[0097] Optionally, the data acquisition system may further include: a third scanning device.

[0098] Figure 5 This is a schematic diagram of the structure of a third scanning device provided in an embodiment of the present application. Figure 5 As shown, the third scanning device includes a second line sensor 501 and a third moving device 502 , and the second line sensor 501 is arranged on the side of the third moving device 502 .

[0099] The second line sensor 501 is used to scan the free-form surface and collect second sparse point cloud data of the free-form surface of the material. The third moving device 502 is used to carry and move the material.

[0100] For example, when the price material is placed on the third moving device 502, the free curved surface of the material is placed upward, and the third moving device 502 in the third scanning device can rotate and move the material, and scan the side of the material through the second line sensor 501. Furthermore, second sparse point cloud data can be obtained from the image data of the side collected by the second line sensor. The second sparse point cloud data is point cloud data of the free curved surface contour.

[0101] Optionally, the data acquisition system further includes: a displacement device.

[0102] The displacement device can be used to receive displacement instructions sent by the data processing device 403 to move the material from the second displacement device to the third displacement device, and move the material from the second displacement device 407 to the third displacement device 502. It can also receive displacement instructions sent by the data processing device 403 to move the material from the third displacement device to the first displacement device, and move the material from the third displacement device 502 to the first displacement device 405.

[0103] The data acquisition method provided by the embodiment of the present application is described in detail below using the above-mentioned data acquisition system as an example. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0104] It should be understood that the data acquisition method of the embodiment of the present application can be used in any scenario of acquiring image data of a free-form surface of a material.

[0105] Figure 6 This is a flow chart of a data collection method provided in an embodiment of the present application. Figure 6 As shown in , the data collection method may include the following steps:

[0106] S601: Acquire first sparse point cloud data of a free-form surface of a material.

[0107] The first sparse point cloud data is relatively scattered point cloud data on the free-form surface of the material, and the first sparse point cloud data is data represented based on three-dimensional coordinates.

[0108] For example, the data processing device 403 may obtain first sparse point cloud data obtained by scanning the free-form surface of the material by the point sensor 406 in the first scanning device 401. For example, the data processing device 403 may control the first scanning device 401 to scan the free-form surface of the material, and the first scanning device 401 may obtain a plurality of point cloud data of the free-form surface of the material by scanning with the point sensor 406 and the second moving device 407.

[0109] Figure 7 This is a schematic diagram of a first sparse point cloud data provided in an embodiment of the present application. Figure 7 , which is a projection diagram of first sparse point cloud data collected from a free-form surface. Each scanning point in the figure is point cloud data obtained by scanning with the first scanning device 401.

[0110] S602: Determine a moving path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of the first line sensor.

[0111] The sensing area of the first line sensor 404 is determined by its hardware parameters. The hardware parameters of the first line sensor 404 include scanning line length and scanning range. The scanning range defines the depth range scanned by the first line sensor 404 , and the scanning line length defines the width scanned by the first line sensor 404 .

[0112] The moving path is the moving path of the material during the scanning process of the first line sensor 404. The moving path may include movement in the horizontal direction and the vertical direction.

[0113] Exemplarily, the data processing device 403 can convert the first sparse point cloud data into image data with depth information, where the depth information is information determined by the vertical coordinate axis of each point cloud data in the first sparse point cloud data in the three-dimensional coordinates. Based on the image data obtained from the first sparse point cloud data, the movement path of the material as the height of the free surface changes can be determined.

[0114] For example, when the height of the material is the default height, if it is determined based on the depth information that a portion of the free-form surface is within the scanning area that can be scanned by the first line sensor 404, then it can be determined that when the first line sensor 404 scans this area, the height of the material movement is the default height; if it is determined based on the depth information that a portion of the free-form surface is lower in height than the scanning area that can be scanned by the first line sensor 404, then it can be determined that when the first line sensor 404 scans this area, the height of the material movement is a height adjusted downward from the default height.

[0115] Exemplarily, the data processing device 403 may use the mean of the depth information of the corresponding image data in the area and the difference of the default height as the height of the material movement, or may use the mean of the maximum depth information and the minimum depth information of the depth information of the corresponding image data in the area and the difference of the default height as the height of the material movement.

[0116] S603 , moving the material based on the moving path, and controlling the first line sensor to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0117] The scanned image data is image data obtained by scanning through the first line sensor 404 .

[0118] For example, the data processing device 403 can obtain scanned image data at different heights of the free-form surface through the second scanning device 402. The data processing device 403 can control the first moving device 405 to move the material along the movement path, and control the first line sensor 404 to scan the free-form surface, thereby obtaining image data scanned by the first line sensor 404. Based on the height adjustment of the material, the data processing device 403 can obtain scanned image data at different heights of the free-form surface.

[0119] The data acquisition method of the present application obtains first sparse point cloud data of the free-form surface of the material, determines the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of the first line sensor, moves the material based on the movement path, and controls the line sensor to scan the free-form surface to obtain scanned image data of different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the movement path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of the line sensor, thereby scanning image data of different heights of the free-form surface and obtaining complete image data of the free-form surface.

[0120] The following describes a method for determining the moving path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of the line sensor.

[0121] Optionally, the data processing device may perform point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface, and determine the movement path of the material based on the dense point cloud data and the sensing area of the first line sensor.

[0122] Compared with the first sparse point cloud data, the dense point cloud data has a larger data volume and a smaller point spacing.

[0123] Figure 8The present invention provides a flow chart of a method for generating dense point cloud data of a free-form surface. Figure 8 As shown, performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface may include the following steps:

[0124] S801: Acquire an edge contour of a free-form surface according to first sparse point cloud data.

[0125] Exemplarily, the data processing device 403 can extract the boundary point cloud of the first sparse point cloud data based on the edge detection method in the related art, connect the detected boundary point cloud into an initial edge contour, smooth the initial edge contour, and obtain the edge contour of the free surface.

[0126] Exemplarily, the data processing device 403 may preprocess the first sparse point cloud data to obtain preprocessed first sparse point cloud data, and fit the edge contour of the free-form surface based on the boundary point cloud of the preprocessed first sparse point cloud data.

[0127] In one possible implementation, preprocessing includes noise reduction processing. The data processing device 403 preprocesses the first sparse point cloud data to obtain preprocessed first sparse point cloud data. The first sparse point cloud data can be filtered by sequentially performing through filtering and statistical filtering.

[0128] Through filtering is a filtering method that removes points in point cloud data that are beyond the specified range. The retained range is limited by setting the maximum and minimum values, as shown in formula (1):

[0129] Formula (1)

[0130] in, is any point cloud in the first sparse point cloud data, is the maximum value set. is the minimum value set. The point cloud that satisfies formula (1) is a point cloud that exceeds the specified range. The point cloud that exceeds the specified range is deleted from the first sparse point cloud data.

[0131] Statistical filtering, as shown in formula (2):

[0132] Formula (2)

[0133] in, is the average distance from any point cloud in the first sparse point cloud data to other point clouds in its neighborhood, is the mean of the average distances of all point clouds in the neighborhood, is the standard deviation of the mean distances of all point clouds in the neighborhood, Is the threshold coefficient set to determine the multiple of the standard deviation. The point cloud that satisfies formula (2) is a noise point, and the noise point is deleted from the first sparse point cloud data.

[0134] One possible implementation method is that the preprocessing includes data completion processing. The data processing device 403 preprocesses the first sparse point cloud data to obtain preprocessed first sparse point cloud data. Boundary point cloud data can be extracted from the second sparse point cloud data. The second sparse point cloud data is data collected by the second line sensor 501 set on the side of the free-form surface. The boundary point cloud data is used to supplement the first sparse point cloud data. The first sparse point cloud data is data collected by the point sensor set above the free-form surface.

[0135] For example, the data processing device 403 rotates the material one revolution using the third moving device 502 and controls the second line sensor 501 to scan and obtain image data of the side surface of the material. Boundary point cloud data representing the contour of one side of the free-form surface is extracted from the obtained side surface image data, i.e., the second sparse point cloud data. The first sparse point cloud data is supplemented with the second sparse point cloud data to obtain pre-processed first sparse point cloud data.

[0136] Based on the pre-processed first sparse point cloud data, a boundary point cloud is extracted, and the boundary point cloud is fitted into a curve, which is the edge contour of the free-form surface.

[0137] In this way, by preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data, the quality of the first sparse point cloud can be improved, so that the boundary point cloud based on the preprocessed first sparse point cloud data can fit the edge contour of the free-form surface more accurately.

[0138] S802: Perform surface fitting based on the first sparse point cloud data and the edge contour to obtain a surface function expression of the free-form surface.

[0139] Exemplarily, the data processing device 403 may input the first sparse point cloud data and the edge contour into the surface fitting model through a surface fitting model in related art to obtain a fitted surface function expression.

[0140] For example, the data processing device 403 can obtain the surface function expression of the free-form surface by using a surface fitting algorithm in related technologies. Taking the Non-Uniform Rational B-Splines (NURBS) algorithm as an example, as shown in formula (3):

[0141] Formula (3)

[0142] in, is a point on the NURBS surface, u and v are parameters in the parameter space, n and are the number of control points of the control grid in the u direction and v direction respectively, p and q are the orders of the B-spline basis functions, and is the B-spline basis function defined on the knot vector, It is a control point The weight of are the control points on the control grid, and the edge contours define the range of the control points.

[0143] The NURBS algorithm adjusts the control points and weights by minimizing the error function to make the NURBS surface as close to the point cloud as possible. For example, the NURBS algorithm uses the least squares method to optimize and minimize the error function. The error function can be the sum of the squares of the distances from the point to the surface, as shown in Equation (4):

[0144] Formula (4)

[0145] in, It is the corresponding point of the point cloud Qk in the first sparse point cloud data in the NURBS surface.

[0146] S803 . Fill the first sparse point cloud data based on the surface function expression to obtain dense point cloud data.

[0147] Based on the surface function expression, multiple point cloud data can be determined and then added to the first sparse point cloud data to obtain dense point cloud data. For example, new point cloud data can be inserted between any adjacent point clouds in the first sparse point cloud data using an interpolation method, and the values of the new point cloud data are obtained based on the surface function expression.

[0148] The data processing method of the embodiment of the present application can increase the data volume of the first sparse point cloud data by performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free surface, and provide data support for determining the movement path of the material based on the dense point cloud data and the sensing area of the first-line sensor, so as to make the determined movement path more accurate.

[0149] Exemplarily, based on the dense point cloud data and the sensing area of the line sensor, the movement path of the material is determined. The data processing device 403 can convert the dense point cloud data into first image data with depth information, and determine the movement path of the material based on the first image data and the sensing area of the line sensor.

[0150] Exemplarily, the data processing device 403 may convert the dense point cloud data into image data through projection, and retain the values of the vertical coordinate axis of the dense point cloud data as depth information.

[0151] Optionally, if the coordinate system of the first image data is different from the coordinate system where the first line sensor is located, coordinate transformation is performed on the first image data based on the coordinate system where the first line sensor is located.

[0152] For example, the coordinate system of the first image data is data based on the coordinate system of the point sensor 406. When the scanning range of the point sensor 406 is x and the scanning range of the first line sensor 404 is y, the depth information of the first image data can be increased by the difference between y and x.

[0153] Figure 9 A schematic diagram of a process for determining the movement path of a material provided in an embodiment of the present application. Figure 9 As shown, determining the moving path of the material based on the first image data and the sensing area of the line sensor may include the following steps:

[0154] S901: Determine an initial moving path of the material at a default height based on the first image data and the field of view width corresponding to the sensing area.

[0155] The default height is the preset material height, refer to Figure 4 As shown in the first moving device 405 of the second scanning device 402 , the default height of the material can be regarded as the default height of the first moving device 405 .

[0156] The initial moving path is the moving path of the material in the horizontal direction.

[0157] Exemplarily, the data processing device 403 may determine the initial moving path of the material at the default height according to the bow-shaped path based on the image data and the field of view width corresponding to the sensing area as the path width, and the initial moving path covers the first image data.

[0158] Exemplarily, the data processing device 403 can use the field of view width corresponding to the sensing area of the image data as the path width, starting from the image data at the leftmost boundary of the first image data, from top to bottom as the moving path of the first column, and moving right to the image data at the bottommost boundary of the second column, from bottom to top as the moving path of the second column, and moving right to the image data at the topmost boundary of the third column, according to this planning method, until the first image data is covered, the moving paths of each column and the moving paths between two adjacent columns are connected to determine as the initial moving path.

[0159] Optionally, adjacent scanning columns may have overlapping areas. Taking the initial moving path as an arc-shaped path as an example, for example, adjacent straight line segments may overlap by 20%. Figure 10 Schematic diagram of a scanning column of a starting moving path provided in an embodiment of the present application. Figure 10As shown, the movement paths in each scanning column are represented by arrows, and adjacent columns may partially overlap.

[0160] S902: According to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area, adjust the height of the initial moving path to obtain the moving path.

[0161] Exemplarily, the initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface. Figure 11 A schematic diagram of a moving path provided in an embodiment of the present application Figure 1 .like Figure 11 As shown, the left boundary of the rectangular frame is a straight line segment path, and the area shown in the rectangular frame is a scanning column corresponding to the straight line segment path. The width of the scanning column is determined by the scanning line length of the first line sensor 404.

[0162] The data processing device 403 can obtain the height distribution of each scanning column based on the depth information of the scanning column in the first image data, and adjust the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path.

[0163] Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path. A possible implementation method is that if the height distribution of the scanning column is all within the scanning range, a first height is determined based on the height distribution of the scanning column, and the height of the initial moving path is adjusted using the first height to obtain the moving path.

[0164] Reference Figure 11 As shown, for example, data processing device 403 can read the depth information of the first image data in the scan column and obtain the height distribution of the scan column based on a data distribution algorithm. If the heights of the first image data in the scan column are all within the scanning range, the average of the heights of the first image data in the scan column is used as the first height, and the height of the initial movement path is adjusted using the first height. For example, if the default height is 10 mm and the first height is 15 mm, then when moving based on the default height, the material can be moved vertically by 5 mm.

[0165] In one possible implementation, if the height distribution exceeds the scanning range, the height of the straight line segment path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

[0166] For example, the data processing device 403 may calculate the height distribution from the starting row to the current row of the scan column. If the height distribution of the current row exceeds the scan range, the current row is used as a discontinuity point to obtain the second height of the straight line path segment, and the height of the straight line path segment is adjusted using the second height. The current row is used as the new starting row to obtain the next discontinuity point, and the process continues until the last row of the scan column is reached.

[0167] Figure 12 A schematic diagram of a process for determining a second height is provided in an embodiment of the present application. Figure 12 As shown, for any scan column, it specifically includes:

[0168] S1201. Set the current row as the starting row.

[0169] S1202: Determine whether the current row height distribution exceeds the scanning range.

[0170] If yes, execute S1204;

[0171] If not, execute S1203.

[0172] S1203 : Obtain height distribution of depth information of the first image data of the next row.

[0173] After obtaining the height distribution of the depth information of the first image data of the next row, the process continues with S1202 .

[0174] S1204: Use the current line as a discontinuity point to obtain the second height of the straight line path segment.

[0175] A straight line path segment is from the starting row to the row before the current row.

[0176] Exemplarily, the data processing device 403 may use the average of the depth information of the first image data from the starting row of the scan column to the row before the current row as the second height of the straight line path segment. Alternatively, the data processing device 403 may also determine the maximum and minimum values of the depth information of the first image data from the starting row of the scan column to the row before the current row, and use the average of the maximum and minimum values as the second height of the straight line path segment.

[0177] The current row is used as a discontinuity point, and after obtaining the second height of the straight line segment, the process continues with S1201 until the current row becomes the last row in the scan column, and then processes the remaining scan columns one by one.

[0178] Based on the first height or the second height obtained in the above embodiment, the height of the initial moving path is adjusted using the first height or the second height to obtain the moving path. Figure 13 A schematic diagram of a moving path provided in an embodiment of the present application Figure 2 .like Figure 13As shown, each point is the position of a row of the scanning column, each column is a straight line path, the first column on the far right includes two points, that is, the moving path of the first column has not changed in height, the fourth column includes 4 points, that is, the straight line path segment from the first point to the second point has a height, the straight line path segment from the second point to the third point has a height, and the straight line path segment from the third point to the fourth point has a height, that is, in the process of scanning the free curve surface where the straight line path is located, the height of the material needs to be adjusted once at the second point and the third point respectively.

[0179] The data processing device 403 moves the material based on the moving path, and controls the first line sensor 404 to scan the free-form surface to obtain scanning image data at different heights of the free-form surface.

[0180] The data acquisition method of the present application obtains first sparse point cloud data of the free-form surface of the material, determines the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and the sensing area of the first line sensor, moves the material based on the movement path, and controls the line sensor to scan the free-form surface to obtain scanned image data of different heights of the free-form surface. The method of the present application controls the movement of the material as the height of the free-form surface changes based on the movement path during the process of the first line sensor scanning the free-form surface of the material, so that the free-form surface scanned by the line sensor is within the sensing area of the line sensor, thereby scanning image data of different heights of the free-form surface and obtaining complete image data of the free-form surface.

[0181] Optionally, the data processing device 403 may further splice the scanned image data of the free-form surface based on the obtained scanned image data of the free-form surface at different heights to obtain image data of the free-form surface.

[0182] Exemplarily, the data processing device 403 may stitch the scanned image data based on the starting and ending points of the paths corresponding to the scanned image data to obtain an initial stitched image, and fuse the overlapping parts of adjacent scanned image data in the initial stitched image to obtain image data.

[0183] Based on the starting and ending points of the path corresponding to the scanned image data, the scanned image data is spliced to obtain an initial spliced image. The data processing device 403 can determine the splicing position of each scanned image based on the starting and ending points of the path corresponding to the scanned image data, and splice each scanned image based on the image splicing method according to the splicing position of each scanned image to obtain an initial spliced image.

[0184] In this way, the scanned image data can be quickly spliced to obtain the initial image data of the free-form surface.

[0185] The data of the overlapping parts in the adjacent scanned image data in the initial stitched image are fused to obtain image data. The data processing device 403 can convert the data of the overlapping parts into point cloud data, perform local alignment on the point cloud data, and obtain a transformation matrix of the data of the overlapping parts. Based on the transformation matrix, the data of the overlapping parts are fused to obtain image data.

[0186] Specifically, the two sets of overlapping data are converted into point cloud data, local registration is performed based on the feature matching method, the transformation matrix of the two sets of point cloud data is calculated, one set of point cloud data is transformed according to the transformation matrix, the converted point cloud data is aligned with the other set of point cloud data, and fusion processing is performed to complete the precise stitching of the image containing the overlapping data and obtain the image data of the free-form surface of the material.

[0187] In the method of the embodiment of the present application, the scanned image data is spliced based on the starting and ending points of the path corresponding to the scanned image data to obtain an initial spliced image, and the data of the overlapping parts of the adjacent scanned image data in the initial spliced image are fused to obtain image data. The image data is image data of the free-form surface of the material after more precise splicing.

[0188] The above is an explanation of the method embodiment of the present application. The data processing device 403 provided in the embodiment of the present application can execute the above method embodiment.

[0189] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 14 As shown, the electronic device may include: at least one processor 1401 and a memory 1402.

[0190] The memory 1402 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0191] The memory 1402 may include a high-speed RAM memory, and may also include a non-volatile memory.

[0192] Processor 1401 is configured to execute computer-executable instructions stored in memory 1402 to implement the method of the aforementioned method embodiment. Processor 1401 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0193] Optionally, the electronic device may further include a communication interface 1403. In a specific implementation, if the communication interface 1403, the memory 1402 and the processor 1401 are implemented independently, the communication interface 1403, the memory 1402 and the processor 1401 may be interconnected via a bus and communicate with each other.

[0194] Optionally, in a specific implementation, if the communication interface 1403, the memory 1402 and the processor 1401 are integrated on a chip, the communication interface 1403, the memory 1402 and the processor 1401 can complete communication through an internal interface.

[0195] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, which are used to implement the actions of the above-mentioned method implementation method.

[0196] The present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the actions of the above method implementation are implemented.

[0197] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0198] If an integrated unit / module is implemented in hardware, the hardware may be digital circuits, analog circuits, or other similar devices. The physical implementation of the hardware structure includes, but is not limited to, transistors. Unless otherwise specified, a processor may be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, or ASIC. Unless otherwise specified, a storage unit may be any suitable storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), or hybrid memory cube (HMC).

[0199] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program code, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), and mobile hard drives.

[0200] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0201] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented by hardware associated with program instructions. The aforementioned program can be stored in any computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM and RAM.

Claims

1. A data collection method, characterized in that: include: Acquire first sparse point cloud data of the free-form surface of the material; determining a movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data and a sensing area of the first line sensor; Moving the material based on the moving path, and controlling the first line sensor to scan the free-form surface to obtain scanning image data at different heights of the free-form surface; The determining, based on the first sparse point cloud data and the sensing area of the first line sensor, the moving path of the material as the height of the free-form surface changes, includes: Performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface; determining a moving path of the material based on the dense point cloud data and a sensing area of the first line sensor; The determining the moving path of the material based on the dense point cloud data and the sensing area of the first line sensor includes: Converting the dense point cloud data into first image data with depth information; determining a moving path of the material according to the first image data and a sensing area of the first line sensor; The determining the moving path of the material according to the first image data and the sensing area of the first line sensor includes: determining an initial moving path of the material at a default height based on the first image data and a field of view width corresponding to the sensing area, where the default height is a preset height of the material; The height of the initial moving path is adjusted according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path.

2. The method according to claim 1, characterized in that The step of performing point cloud filling on the first sparse point cloud data to generate dense point cloud data of the free-form surface includes: Acquire an edge contour of the free-form surface according to the first sparse point cloud data; Performing surface fitting based on the first sparse point cloud data and the edge contour to obtain a surface function expression of the free-form surface; The first sparse point cloud data is filled based on the surface function expression to obtain the dense point cloud data.

3. The method according to claim 2, characterized in that The step of obtaining the edge contour of the free-form surface according to the first sparse point cloud data includes: Preprocessing the first sparse point cloud data to obtain preprocessed first sparse point cloud data; Based on the boundary point cloud of the preprocessed first sparse point cloud data, an edge contour of the free-form surface is fitted.

4. The method according to claim 3, characterized in that The preprocessing includes noise reduction processing, and the preprocessing of the first sparse point cloud data to obtain the preprocessed first sparse point cloud data includes: The first sparse point cloud data is sequentially subjected to through-filtering and statistical filtering to obtain filtered first sparse point cloud data.

5. The method according to claim 3, characterized in that The preprocessing includes data completion processing, and the preprocessing of the first sparse point cloud data to obtain preprocessed first sparse point cloud data includes: Extracting boundary point cloud data from second sparse point cloud data, where the second sparse point cloud data is data collected by a second line sensor disposed on a side surface of the free-form surface; The first sparse point cloud data is supplemented by using the boundary point cloud data, where the first sparse point cloud data is data collected by a point sensor disposed above the free-form surface.

6. The method according to any one of claims 1 to 5, characterized in that The initial moving path is a bow-shaped path, and a straight line segment path on the bow-shaped path corresponds to a scanning column of the free-form surface; Adjusting the height of the initial moving path according to the initial moving path, the depth information in the first image data, and the scanning range corresponding to the sensing area to obtain the moving path includes: For each scanning column, acquiring a height distribution of the scanning column based on depth information of the scanning column in the first image data; Based on the height distribution of the scanning column and the scanning range, the height of the initial moving path is adjusted to obtain the moving path.

7. The method according to claim 6, characterized in that The step of adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes: If the height distributions of the scanning rows are all located within the scanning range, determining a first height based on the height distributions of the scanning rows; The height of the initial moving path is adjusted using the first height to obtain the moving path.

8. The method according to claim 6, characterized in that The step of adjusting the height of the initial moving path based on the height distribution of the scanning column and the scanning range to obtain the moving path includes: If the height distribution exceeds the scanning range, the height of the straight path is adjusted based on the height of each row of the scanning column and the scanning range corresponding to the sensing area.

9. The method according to claim 8, characterized in that The adjusting the height of the straight path based on the height of each row of the scanning column and the scanning range corresponding to the sensing area includes: Calculating the height distribution from the starting row to the current row of the scan column; if the height distribution of the current row exceeds the scan range, using the current row as a discontinuity point, obtaining a second height of the straight line path segment, and adjusting the height of the straight line path segment using the second height; The current row is used as a new starting row to obtain the next breakpoint until the last row of the scan column is traversed.

10. The method according to claim 6, characterized in that Before determining the moving path of the material according to the first image data and the sensing area of the first line sensor, the method further includes: If the coordinate system of the first image data is different from the coordinate system of the first line sensor, coordinate conversion is performed on the first image data based on the coordinate system of the first line sensor.

11. The method according to any one of claims 1 to 5, characterized in that Also includes: The scanned image data of the free-form surface are spliced to obtain image data of the free-form surface.

12. The method according to claim 11, characterized in that The step of stitching the scanned image data of the free-form surface to obtain the image data of the free-form surface includes: splicing the scanned image data based on the path start and end points corresponding to the scanned image data to obtain an initial spliced image; The image data is obtained by performing data fusion processing on the overlapping parts of the adjacent scanned image data in the initial spliced image.

13. The method according to claim 12, characterized in that The data fusion processing of the overlapping parts of the adjacent scanned image data in the initial spliced image to obtain the image data includes: Converting the data of the overlapping part into point cloud data; performing local registration on the point cloud data to obtain a transformation matrix of the data of the overlapping part; Based on the transformation matrix, the data of the overlapping parts are fused to obtain the image data.

14. A data acquisition system, characterized in that: include: a first scanning device, a second scanning device, and a data processing device; The first scanning device is used to collect first sparse point cloud data of the free-form surface of the material; The second scanning device includes a first line sensor and a first moving device, wherein the first line sensor is arranged above the first moving device; the first moving device is used to carry materials; The data processing device is used to determine the movement path of the material as the height of the free-form surface changes based on the first sparse point cloud data using the method described in any one of claims 1 to 13, and control the first moving device to move the material based on the movement path, and control the first line sensor to scan the free-form surface to obtain scanning image data of the free-form surface at different heights.

15. The system according to claim 14, wherein: The first scanning device includes a point sensor and a second moving device, and the point sensor is arranged above the second moving device; The point sensor is used to scan the free-form surface and collect first sparse point cloud data of the free-form surface of the material; The second moving device is used to carry and move the material.

16. The system according to claim 14 or 15, characterized in that The system further includes: a third scanning device, the third scanning device including a second line sensor and a third moving device; the second line sensor is disposed on a side of the third moving device; The second line sensor is used to scan the free-form surface and collect second sparse point cloud data of the free-form surface of the material; The third moving device is used to carry and move the material.

17. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 13.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 13 when executed by a processor.

19. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 13 when being executed by a processor.

Images