Method and device for processing point cloud data

Abstract

The present invention discloses a method and apparatus for processing point cloud data, relating to the field of image processing technology. A specific implementation of the method includes: acquiring point cloud data of an object, projecting the point cloud data onto the plane where the object surface is located along the normal of the object surface to obtain the projection outline of the object; uniformly sampling points on the projection outline to obtain N points of the projection outline; calculating a set of right-angled vertices based on the N points; dividing the plane where the object surface is located into a plurality of grids, calculating the local maximum of the number of right-angled vertices falling within the plurality of grids, thereby screening out the target grid corresponding to the local maximum; and connecting the geometric centers of the right-angled vertices in each target grid to form a closed right-angled polygon. This implementation can solve the technical problem that fluctuations and depressions in the projection outline affect the calculation of right-angled polygons.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for processing point cloud data. Background Art

[0002] In many scenarios within unmanned warehouses, item detection and positioning are required to enable automated picking. For example, a turnover box running on a conveyor belt stops at a designated location to pick the contents, or stacked items are placed at a designated workstation awaiting unloading. This typically requires the aid of a vision system, which uses two-dimensional or three-dimensional visual information to segment the items and then locate them at the appropriate location for grabbing. To achieve more accurate item positioning, it is crucial to quickly and accurately calculate the vertices of the rectangular polygons formed by the point cloud data.

[0003] In the process of implementing the present invention, the inventors discovered that the prior art has at least the following problems:

[0004] Items are typically packaged in square boxes, so theoretically their outline should form a rectangular polygon. However, depending on the quality of the point cloud data, the resulting projected outline can vary. In addition to small fluctuations in the boundary, poor outlines can also have large depressions caused by holes in the missing points. These fluctuations and depressions can affect the calculation of the rectangular polygon, leading to inaccurate object positioning. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a method and apparatus for processing point cloud data to solve the technical problem that fluctuations and depressions of projection contours affect the calculation of right-angled polygons.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for processing point cloud data is provided, comprising:

[0007] Acquiring point cloud data of an object, and projecting the point cloud data onto a plane on which the surface of the object is located along a normal line of the surface of the object to obtain a projected outline of the object;

[0008] Evenly selecting points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetic progression; calculating a set of right-angle vertices based on the N points, where N is a preset number threshold;

[0009] Dividing the plane where the surface of the object is located into a plurality of grids, calculating the local maximum of the number of right-angle vertices falling in the plurality of grids, and thereby screening out the target grid corresponding to the local maximum;

[0010] The geometric centers of each right-angled vertex in the target mesh are connected to form a closed right-angled polygon.

[0011] Optionally, evenly selecting points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetic progression, includes:

[0012] Calculating the distance from each point in the point sequence of the projection contour to the starting point of the projection contour;

[0013] For each point in the point sequence, a fixed ratio is calculated according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold;

[0014] If the fixed ratio is greater than or equal to zero, the coordinates of the point are calculated based on the fixed ratio, the coordinates of the point and the coordinates of the previous point of the point.

[0015] Optionally, calculating the fixed ratio according to the distance from the point to the starting point of the projection outline, the total length of the projection outline, the number of currently calculated points, and a preset number threshold includes:

[0016] Dividing the distance from the point to the starting point of the projected contour by the total length of the projected contour to obtain a first quotient;

[0017] Dividing the number of points currently calculated by a preset number threshold to obtain a second quotient;

[0018] Subtract the first quotient from the second quotient to obtain a fixed ratio.

[0019] Optionally, calculating the coordinates of the point according to the fixed ratio, the coordinates of the point, and the coordinates of the point before the point includes:

[0020] Multiplying the fixed ratio by the coordinates of the previous point of the point to obtain a first coordinate;

[0021] Multiplying the difference of 1 minus the fixed ratio by the coordinate of the point to obtain a second coordinate;

[0022] The first coordinate is added to the second coordinate to obtain the coordinates of the point.

[0023] Optionally, calculating a set of right-angle vertices based on the N divided points includes:

[0024] For each of the N point points, calculating an angle between a first straight line passing through the point point and a point next to the point point, and a second straight line passing through the other point point and a point next to the other point point;

[0025] If the absolute value of the included angle minus 90° is less than a preset angle threshold, the intersection of the first straight line and the second straight line is taken as a right-angle vertex, and the right-angle vertex is added to the right-angle vertex set.

[0026] Optionally, dividing the plane where the surface of the object is located into a plurality of grids, calculating the local maximum of the number of right-angled vertices falling in the plurality of grids, and thereby screening out the target grid corresponding to the local maximum, includes:

[0027] The plane where the surface of the object is located is evenly divided into a plurality of grids, and the number of right-angled vertices falling in each grid is calculated respectively;

[0028] A local maximum value of the number of right-angle vertices in the grid is searched, and a grid corresponding to the local maximum value is used as a target grid.

[0029] Optionally, searching for a local maximum value of the number of right-angle vertices in the grid and using the grid corresponding to the local maximum value as the target grid includes:

[0030] Searching for a local maximum in the number of right-angle vertices in the grid;

[0031] Performing non-maximum suppression on the grid corresponding to the local maximum;

[0032] The grid corresponding to the local maximum value after non-maximum value suppression is used as the target grid.

[0033] Optionally, connecting the geometric centers of the right-angled vertices in each of the target meshes into a closed right-angled polygon comprises:

[0034] Calculating the geometric center of each right-angle vertex in the target grid respectively;

[0035] In the order in which the right-angled vertices are placed in the right-angled vertex set, each of the geometric centers is connected to form a closed right-angled polygon.

[0036] Optionally, before uniformly selecting points for the projection contour, the method further includes:

[0037] According to a preset step size or window size, mean filtering is performed on the point sequence of the projection contour.

[0038] In addition, according to another aspect of an embodiment of the present invention, there is provided an apparatus for processing point cloud data, comprising:

[0039] a projection module, configured to acquire point cloud data of an object and project the point cloud data onto a plane on which the surface of the object is located along a normal line of the surface of the object to obtain a projection outline of the object;

[0040] a processing module configured to uniformly select points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetically; and calculate a set of right-angle vertices based on the N points, wherein N is a preset number threshold;

[0041] a screening module, configured to divide the plane on which the surface of the object is located into a plurality of grids, calculate the local maximum of the number of right-angled vertices falling in the plurality of grids, and thereby screen out a target grid corresponding to the local maximum;

[0042] The connection module is used to connect the geometric centers of the right-angle vertices in each target grid into a closed right-angle polygon.

[0043] Optionally, the processing module is further configured to:

[0044] Calculating the distance from each point in the point sequence of the projection contour to the starting point of the projection contour;

[0045] For each point in the point sequence, a fixed ratio is calculated according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold;

[0046] If the fixed ratio is greater than or equal to zero, the coordinates of the point are calculated based on the fixed ratio, the coordinates of the point and the coordinates of the previous point of the point.

[0047] Optionally, the processing module is further configured to:

[0048] Dividing the distance from the point to the starting point of the projected contour by the total length of the projected contour to obtain a first quotient;

[0049] Dividing the number of points currently calculated by a preset number threshold to obtain a second quotient;

[0050] Subtract the first quotient from the second quotient to obtain a fixed ratio.

[0051] Optionally, the processing module is further configured to:

[0052] Multiplying the fixed ratio by the coordinates of the previous point of the point to obtain a first coordinate;

[0053] Multiplying the difference of 1 minus the fixed ratio by the coordinate of the point to obtain a second coordinate;

[0054] The first coordinate is added to the second coordinate to obtain the coordinates of the point.

[0055] Optionally, the processing module is further configured to:

[0056] For each of the N point points, calculating an angle between a first straight line passing through the point point and a point next to the point point, and a second straight line passing through the other point point and a point next to the other point point;

[0057] If the absolute value of the included angle minus 90° is less than a preset angle threshold, the intersection of the first straight line and the second straight line is taken as a right-angle vertex, and the right-angle vertex is added to the right-angle vertex set.

[0058] Optionally, the screening module is further configured to:

[0059] The plane where the surface of the object is located is evenly divided into a plurality of grids, and the number of right-angled vertices falling in each grid is calculated respectively;

[0060] A local maximum value of the number of right-angle vertices in the grid is searched, and a grid corresponding to the local maximum value is used as a target grid.

[0061] Optionally, the screening module is further configured to:

[0062] Searching for a local maximum in the number of right-angle vertices in the grid;

[0063] Performing non-maximum suppression on the grid corresponding to the local maximum;

[0064] The grid corresponding to the local maximum value after non-maximum value suppression is used as the target grid.

[0065] Optionally, the connection module is further configured to:

[0066] Calculating the geometric center of each right-angle vertex in the target grid respectively;

[0067] In the order in which the right-angled vertices are placed in the right-angled vertex set, each of the geometric centers is connected to form a closed right-angled polygon.

[0068] Optionally, the processing module is further configured to:

[0069] Before uniformly selecting points for the projection contour, mean filtering is performed on the point sequence of the projection contour according to a preset step size or window size.

[0070] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:

[0071] one or more processors;

[0072] a storage device for storing one or more programs,

[0073] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.

[0074] According to another aspect of the embodiments of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0075] One embodiment of the above invention has the following advantages or beneficial effects: because the projection contour is uniformly selected to obtain N points of the projection contour, the distance from each point to the starting point of the projection contour is successively and arbitrarily increased, and then a set of right-angled vertices is calculated based on the N points, and finally the target grid corresponding to the local maximum is screened, and the geometric centers of the right-angled vertices in each target grid are connected to form a closed right-angled polygon. Therefore, the technical problem in the prior art that the fluctuation and concavity of the projection contour affect the calculation of the right-angled polygon is overcome. The embodiment of the present invention can quickly and accurately calculate the right-angled polygon of the projection contour, greatly reducing the interference caused by factors such as image texture, lighting, color, and geometric size in subsequent matching, which helps to accurately match and locate objects; and the embodiment of the present invention has good robustness.

[0076] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0078] Figure 1 is a schematic diagram of the main process of a method for processing point cloud data according to an embodiment of the present invention;

[0079] Figure 2 is a schematic diagram of a robot box picking operation scenario according to an embodiment of the present invention;

[0080] Figure 3a and 3b is a schematic diagram of a projection profile according to an embodiment of the present invention;

[0081] Figure 4a and 4b is a schematic diagram of a right-angled polygon according to an embodiment of the present invention;

[0082] Figure 5 is a schematic diagram of the main process of a method for processing point cloud data according to a reference embodiment of the present invention;

[0083] Figure 6is a schematic diagram of the main process of a method for processing point cloud data according to another reference embodiment of the present invention;

[0084] Figure 7 is a schematic diagram of main modules of an apparatus for processing point cloud data according to an embodiment of the present invention;

[0085] Figure 8 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0086] Figure 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0087] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0088] Figure 1 Schematic diagram of the main process of the method for processing point cloud data according to an embodiment of the present invention. As an embodiment of the present invention, Figure 1 As shown, the method for processing point cloud data may include:

[0089] Step 101 : Acquire point cloud data of an object, and project the point cloud data onto a plane where the object surface is located along a normal line of the object surface to obtain a projection outline of the object.

[0090] like Figure 2 As shown, a camera can be used to collect point cloud data of items in a turnover box. The turnover box is generally made up of square (rectangular and / or square) boxes of the same size stacked layer by layer. Point cloud data is intercepted along the normal of each layer, and then the point cloud data is projected onto the surface of the items to obtain the projection outline of each item in the turnover box, as shown in FIG. Figure 3a and Figure 3b shown.

[0091] It should be noted that the normal is theoretically perpendicular to the surface of the object being imaged, and is generally also perpendicular to the ground. However, if there is an inclination between the turnover box and the ground, the direction perpendicular to the surface of the object (i.e., the normal) is not perpendicular to the ground.

[0092] like Figure 3a and Figure 3b As shown in Figure 2, due to the quality of the point cloud data, the contours obtained by projection are also good or bad. Figure 3a The projected outline shown here exhibits minor fluctuations along the poor outline boundary and large depressions due to holes caused by missing point clouds. These fluctuations and depressions complicate the calculation of the right-angled polygon, leading to inaccurate object positioning. Therefore, this embodiment of the present invention requires further processing of the projected outline (steps 102-104) to obtain a right-angled polygon.

[0093] Step 102 : Evenly select points on the projection contour to obtain N points of the projection contour, so that the distance from each point to the starting point of the projection contour increases arithmetic progression; and calculate a set of right-angle vertices based on the N points.

[0094] After obtaining the projection outline of the point cloud data, points are evenly spaced to form the projection outline, ensuring that the distance from each point to the starting point of the projection outline increases in arithmetic progression. Assuming the starting point of the projection outline is O and the points are A, B, and C, then the lengths of OA, OB, and OC increase in arithmetic progression. After obtaining N points, several right-angled vertices are further calculated based on the Hough voting principle to form a right-angled vertex set.

[0095] Optionally, to remove noise from the projection profile and smooth the projection profile, before step 102, the method further includes performing mean filtering on the projection profile according to a preset step size or window size. In an embodiment of the present invention, the step size or window size can be preset, and then mean filtering is performed on the sequence of points in the projection profile according to the preset step size or window size to obtain a smooth projection profile. w can be preset to 3, 5, 6, 7, 8, or 9, and the window size can be set to 1 mm*1 mm, 2 mm*2 mm, or 1 cm*1 cm, etc.

[0096] Taking the preset step size as an example, assuming that the point sequence constituting the projection contour is C:c0,c1,…,c n-1 ;

[0097] Among them, each point in the point sequence is represented by a two-dimensional vector, that is, c i =(x i ,y i ).

[0098] Next, use the following formula to perform mean filtering on each point:

[0099]

[0100] Wherein, w is a positive integer smaller than n. Optionally, w can be pre-set to 3, 5, 6, 7, 8 or 9, etc., which is not limited in the embodiment of the present invention.

[0101] Finally, update the point sequence C.

[0102] It should be noted that mod(i+k,n) is the remainder of i+k divided by n. For example, mod(7,3) equals 1, and mod(12,5) equals 2. Point addition is vector addition, that is, adding the coordinates of the points. This embodiment of the present invention performs mean filtering on the points. The purpose of using modulo filtering is to process the boundaries of the point sequence at the beginning and end, which can be considered as cyclic boundary conditions.

[0103] Optionally, evenly selecting points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetically, including: calculating the distance from each point in the point sequence of the projection contour to the starting point of the projection contour; for each point in the point sequence, calculating a fixed ratio based on the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of points currently calculated, and a preset threshold value; if the fixed ratio is greater than or equal to zero, calculating the coordinates of the point based on the fixed ratio, the coordinates of the point, and the coordinates of the previous point. The fixed ratio can be accurately calculated based on the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of points currently calculated, and the preset threshold value, thereby accurately calculating the coordinates of the point.

[0104] Alternatively, the Euclidean distance can be used to calculate the distance from each point in the point sequence to the starting point of the projected contour. For example:

[0105] Let l0 = 0, and use the following formula to calculate the distance from each point in the point sequence to the starting point of the projected contour, that is, the cumulative length of the contour curve:

[0106] l i =l i-1 +distance(c i-1 ,c i ),i=1,…,n.

[0107] Among them, distance(c i-1 ,c i ) represents point c i-1 and point c i The Euclidean distance between .

[0108] Optionally, a fixed ratio is calculated based on the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated sub-points, and a preset threshold value, including: dividing the distance from the point to the starting point of the projection contour by the total length of the projection contour to obtain a first quotient; dividing the currently calculated number of sub-points by a preset threshold value to obtain a second quotient; and subtracting the first quotient from the second quotient to obtain the fixed ratio. Optionally, the coordinates of a sub-point are calculated based on the fixed ratio, the coordinates of the point, and the coordinates of the previous point, including: multiplying the fixed ratio by the coordinates of the previous point to obtain a first coordinate; multiplying the difference between 1 and the fixed ratio by the coordinates of the point to obtain a second coordinate; and adding the first coordinate to the second coordinate to obtain the coordinates of the sub-point. The method provided by the embodiments of the present invention can accurately calculate the fixed ratio and the group coordinates of the sub-points.

[0109] For example, the following method can be used to calculate the fixed ratio (λ) and the point (p j ):

[0110] Step 1) Let p0=c0, j=1, i=1;

[0111] Step 2) If j>N, then based on the Hough voting principle, calculate the set of right-angle vertices through N points; otherwise, execute step 3);

[0112] Step 3) Order If λ≥0, then

[0113] p j =λc i-1 +(1-λ)c i

[0114] j=j+1

[0115] Repeat step 2)

[0116] If λ<0, let i=i+1 and repeat step 3) until j>N.

[0117] Wherein, N is a preset quantity threshold. Optionally, the value of N can be 1000-2000.

[0118] Optionally, calculating a set of right-angled vertices based on the N point points includes: for each of the N point points, calculating the angle between a first straight line passing through the point and a point next to the point point, and a second straight line passing through the other point point and a point next to the other point point; if the absolute value of the angle minus 90° is less than a preset angle threshold, determining the intersection of the first straight line and the second straight line as a right-angled vertex, and adding the right-angled vertex to the set of right-angled vertices.

[0119] For example:

[0120] Step 4) Let the right-angle vertex set j=0, k=1

[0121] Step 5) Order

[0122] α = passing point p j and point p j+1 The straight line passing through point p j+k and point p j+k+1 The angle between the straight lines,

[0123] if Then order

[0124] p = passing point p j and point p j+1 The straight line passing through point p j+k and point p j+k+1 The intersection of the straight lines,

[0125] S=S∪{p}

[0126] Step 6) Let k = k + 1

[0127] If k>δ and j>N, then execute step 103; otherwise, let

[0128] j=j+1

[0129] k=1

[0130] Go to step 5)

[0131] Among them, δ is the neighborhood radius, ε is the angle threshold, and δ and ε are pre-set.

[0132] Optionally, the value of δ may be 10-20; the value of ε may be 1°-3°, which are used to determine whether it is a right angle.

[0133] After the above operations, the points in S will be clustered near the right-angle vertex, which is exactly the right-angle vertex we want to find.

[0134] Step 103 : Divide the plane where the surface of the object is located into a plurality of grids, calculate the local maximum values of the number of right-angled vertices falling in the plurality of grids, and thereby select the target grid corresponding to the local maximum value.

[0135] The two-dimensional plane where the surface of the object is located is divided into multiple grids, and the local maximum of the number of right-angled vertices falling in each grid is calculated, thereby screening out the target grid corresponding to the local maximum. Optionally, step 103 may include: evenly dividing the plane where the surface of the object is located into several grids, respectively calculating the number of right-angled vertices falling in each grid; searching for the local maximum of the number of right-angled vertices in the grids, and using the grid corresponding to the local maximum as the target grid. The grids can be used as the X-axis and Y-axis of the two-dimensional histogram, and the number of right-angled vertices in the grids as the Z-axis, so that the local maximum of the histogram surface can be searched. In an embodiment of the present invention, the local maximum refers to the number of right-angled vertices in the target grid.

[0136] Optionally, searching for a local maximum in the number of right-angled vertices in the mesh and using the mesh corresponding to the local maximum as the target mesh may include: searching for a local maximum in the number of right-angled vertices in the mesh; performing non-maximum suppression on the mesh corresponding to the local maximum; and using the mesh corresponding to the non-maximum-suppressed local maximum as the target mesh. To improve the accuracy of the calculation results, non-maximum suppression may be performed on the local maximum points (i.e., the mesh corresponding to the local maximum) and the remaining local maximum points (i.e., the mesh corresponding to the final local maximum) may be output.

[0137] Step 104: Connect the geometric centers of the right-angled vertices in each target mesh to form a closed right-angled polygon.

[0138] After searching for the target mesh, the geometric center of each right-angled vertex in the target mesh is calculated respectively, and then these geometric centers are connected in sequence to obtain a closed right-angled polygon, such as Figure 4a and Figure 4b shown.

[0139] Optionally, step 104 may include: calculating the geometric center of each right-angled vertex in the target mesh; and connecting each of the geometric centers into a closed right-angled polygon in the order in which the right-angled vertices are placed in the right-angled vertex set. To prevent errors in connecting the geometric centers, each geometric center needs to be connected into a closed right-angled polygon in the order in which the right-angled vertices are placed in the right-angled vertex set.

[0140] In practice, a large number of items are packaged in square boxes. Typically, multiple items of the same type are densely packed within a turnover box or on a pallet. The method provided by this invention can calculate rectangular polygons, then match them based on the size (length and width) of the square items, thereby achieving the purpose of segmenting the items. After segmenting the items, they are matched on the two-dimensional image. Using the calibration relationship of a two-dimensional or three-dimensional camera, point cloud data of the items is obtained, and the center and posture of the items are calculated. Finally, the suction cup at the end of the robotic arm grasps the items.

[0141] According to the various embodiments described above, it can be seen that the embodiment of the present invention obtains N points of the projection contour by uniformly selecting points from the projection contour, so that the distance from each point to the starting point of the projection contour increases in arithmetic progression, and then calculates a set of right-angled vertices based on the N points. Finally, the target grid corresponding to the local maximum value is screened out, and the geometric centers of the right-angled vertices in each target grid are connected to form a closed right-angled polygon. This technical means solves the technical problem in the prior art that the fluctuation and concavity of the projection contour affect the calculation of the right-angled polygon. The embodiment of the present invention can quickly and accurately calculate the right-angled polygon of the projection contour, greatly reducing the interference caused by factors such as image texture, lighting, color, and geometric size in subsequent matching, which helps to accurately match and locate objects; and the embodiment of the present invention has good robustness.

[0142] Figure 5 Schematic diagram of the main process of a method for processing point cloud data according to a reference embodiment of the present invention. As another embodiment of the present invention, Figure 5 As shown, the method for processing point cloud data may include:

[0143] Step 501 : Acquire point cloud data of an object, and project the point cloud data onto a plane where the object surface is located along a normal line of the object surface to obtain a projection outline of the object.

[0144] Step 502: Perform mean filtering on the point sequence of the projection contour according to a preset step size or window size.

[0145] The step size or window size can be pre-set, and then the point sequence of the projection contour is subjected to mean filtering according to the set step size or window size to obtain a smooth projection contour. Optionally, the following formula can be used to perform mean filtering on each point:

[0146]

[0147] Where w is a positive integer less than n, n is the number of points in the point sequence, c i For a point in a point sequence, a two-dimensional vector (x i,y i ) to indicate.

[0148] Step 503 : Evenly select points from the projection contour to obtain N points of the projection contour, so that the distance from each point to the starting point of the projection contour increases in arithmetic progression, where N is a preset number threshold.

[0149] The projection contour after mean filtering is evenly divided into points so that the distance from each point to the starting point of the projection contour increases in arithmetic progression. Assuming that the starting point of the projection contour is O and the points are A, B, and C, then the lengths of OA, OB, and OC increase in arithmetic progression.

[0150] Step 504: Based on the Hough voting principle, a set of right-angle vertices is calculated using the N points.

[0151] After obtaining N points, several right-angle vertices are further calculated based on the Hough voting principle to form a right-angle vertex set.

[0152] Step 505: evenly divide the plane where the surface of the object is located into a plurality of grids, and calculate the number of right-angled vertices falling in each grid.

[0153] Step 506: Search for the local maximum value of the number of right-angle vertices in the grid.

[0154] The grid can be used as the X-axis and Y-axis of a two-dimensional histogram, and the number of right-angled vertices in the grid can be used as the Z-axis, so that the local maximum of the histogram surface can be searched. In an embodiment of the present invention, the local maximum refers to the number of right-angled vertices in the target grid.

[0155] Step 507: performing non-maximum suppression on the grid corresponding to the local maximum.

[0156] Step 508: The grid corresponding to the local maximum value after non-maximum value suppression is used as the target grid.

[0157] In order to improve the accuracy of the calculation results, the above local maximum points (i.e., the grids corresponding to the local maximum) can be non-maximum suppressed, and the remaining local maximum points (i.e., the grids corresponding to the final local maximum) can be output.

[0158] Step 509: Connect the geometric centers of the right-angled vertices in each target mesh to form a closed right-angled polygon.

[0159] After searching for the target mesh, the geometric center of each right-angled vertex in the target mesh is calculated respectively, and then these geometric centers are connected in sequence to obtain a closed right-angled polygon.

[0160] In addition, the specific implementation content of the method for processing point cloud data in a reference embodiment of the present invention has been described in detail in the method for processing point cloud data described above, so the repeated content will not be described again here.

[0161] Figure 6 Schematic diagram of the main process of a method for processing point cloud data according to another reference embodiment of the present invention. As another embodiment of the present invention, Figure 6 As shown, the method for processing point cloud data may include:

[0162] Step 601 : Acquire point cloud data of an object, and project the point cloud data onto a plane where the surface of the object is located along a normal line of the surface of the object to obtain a projection outline of the object.

[0163] Step 602: Calculate the distance from each point in the point sequence of the projection contour to the starting point of the projection contour.

[0164] Alternatively, the Euclidean distance can be used to calculate the distance from each point in the point sequence to the starting point of the projected contour. For example:

[0165] Let l0 = 0, and use the following formula to calculate the distance from each point in the point sequence to the starting point of the projected contour, that is, the cumulative length of the contour curve:

[0166] l i =l i-1 +distance(c i-1 ,c i ),i=1,…,n.

[0167] Among them, distance(c i-1 ,c i ) represents point c i-1 and point c i The Euclidean distance between .

[0168] Step 603 : For each point in the point sequence, a fixed ratio is calculated according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold.

[0169] Step 604: If the fixed ratio is greater than or equal to zero, the coordinates of the point are calculated based on the fixed ratio, the coordinates of the point and the coordinates of the previous point of the point.

[0170] For step 603-step 604, the following method can be used to calculate the fixed ratio (λ) and the dividing point (p j ):

[0171] Step 1) Let p0=c0, j=1, i=1;

[0172] Step 2) If j>N, execute step 605; otherwise, execute step 3);

[0173] Step 3) Order If λ≥0, then

[0174] p j =λc i-1 +(1-λ)c i

[0175] j=j+1

[0176] Repeat step 2)

[0177] If λ<0, let i=i+1 and repeat step 3) until j>N.

[0178] Wherein, N is a preset quantity threshold. Optionally, the value of N can be 100-3000.

[0179] Step 605 : For each of the N points, calculate the angle between a first straight line passing through the point and the next point after the point, and a second straight line passing through the other point and the next point after the other point.

[0180] Step 606: If the absolute value of the included angle minus 90° is less than a preset angle threshold, the intersection of the first straight line and the second straight line is taken as a right-angle vertex, and the right-angle vertex is added to a right-angle vertex set.

[0181] For steps 605 and 606, the following method can be used to calculate the right-angled vertex:

[0182] Step 4) Let the right-angle vertex set j=0, k=1

[0183] Step 5) Order

[0184] α = passing point p j and point p j+1 The straight line passing through point p j+k and point p j+k+1 The angle between the straight lines,

[0185] if Then order

[0186] p = passing point p j and point p j+1 The straight line passing through point p j+k and point p j+k+1 The intersection of the straight lines,

[0187] S=S∪{p}

[0188] Step 6) Let k = k + 1

[0189] If k>δ and j>N, then execute step 103; otherwise, let

[0190] j=j+1

[0191] k=1

[0192] Go to step 5)

[0193] Among them, δ is the neighborhood radius, ε is the angle threshold, and δ and ε are pre-set.

[0194] Optionally, the value of δ may be 5-30; the value of ε may be 2°-5°, which are used to determine whether it is a right angle.

[0195] Step 607 : Calculate the local maximum value of the number of right-angled vertices in each grid on the plane where the surface of the object is located, thereby screening out the target grid corresponding to the local maximum value.

[0196] Step 608: Connect the geometric centers of the right-angled vertices in each target mesh to form a closed right-angled polygon.

[0197] To prevent errors when connecting geometric centers, it is necessary to connect each geometric center into a closed right-angled polygon in the order in which the right-angled vertices were placed in the right-angled vertex set. Specifically, the geometric center of each right-angled vertex in the target mesh is calculated separately; then, each geometric center is connected into a closed right-angled polygon in the order in which the right-angled vertices were placed in the right-angled vertex set.

[0198] In addition, the specific implementation content of the method for processing point cloud data in another reference embodiment of the present invention has been described in detail in the method for processing point cloud data described above, so the repeated content will not be described again here.

[0199] Figure 7 Schematic diagram of the main modules of the device for processing point cloud data according to an embodiment of the present invention. Figure 7As shown, the device 700 for processing point cloud data includes a projection module 701, a processing module 702, a screening module 703 and a connection module 704; wherein, the projection module 701 is used to obtain point cloud data of an object, and project the point cloud data onto the plane where the object surface is located along the normal of the object surface to obtain the projection outline of the object; the processing module 702 is used to evenly select points from the projection outline to obtain N points of the projection outline, so that the distance from each point to the starting point of the projection outline increases arithmetic progression; a set of right-angled vertices is calculated based on the N points; wherein N is a preset number threshold; the screening module 703 is used to divide the plane where the object surface is located into several grids, calculate the local maximum of the number of right-angled vertices falling in the several grids, and thereby screen out the target grid corresponding to the local maximum; the connection module 704 is used to connect the geometric centers of the right-angled vertices in each of the target grids into a closed right-angled polygon.

[0200] Optionally, the processing module 702 is further configured to:

[0201] Calculating the distance from each point in the point sequence of the projection contour to the starting point of the projection contour;

[0202] For each point in the point sequence, a fixed ratio is calculated according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold;

[0203] If the fixed ratio is greater than or equal to zero, the coordinates of the point are calculated based on the fixed ratio, the coordinates of the point and the coordinates of the previous point of the point.

[0204] Optionally, the processing module 702 is further configured to:

[0205] Dividing the distance from the point to the starting point of the projected contour by the total length of the projected contour to obtain a first quotient;

[0206] Dividing the number of points currently calculated by a preset number threshold to obtain a second quotient;

[0207] Subtract the first quotient from the second quotient to obtain a fixed ratio.

[0208] Optionally, the processing module 702 is further configured to:

[0209] Multiplying the fixed ratio by the coordinates of the previous point of the point to obtain a first coordinate;

[0210] Multiplying the difference of 1 minus the fixed ratio by the coordinate of the point to obtain a second coordinate;

[0211] The first coordinate is added to the second coordinate to obtain the coordinates of the point.

[0212] Optionally, the processing module 702 is further configured to:

[0213] For each of the N point points, calculating an angle between a first straight line passing through the point point and a point next to the point point, and a second straight line passing through the other point point and a point next to the other point point;

[0214] If the absolute value of the included angle minus 90° is less than a preset angle threshold, the intersection of the first straight line and the second straight line is taken as a right-angle vertex, and the right-angle vertex is added to the right-angle vertex set.

[0215] Optionally, the screening module 703 is further configured to:

[0216] The plane where the surface of the object is located is evenly divided into a plurality of grids, and the number of right-angled vertices falling in each grid is calculated respectively;

[0217] A local maximum value of the number of right-angle vertices in the grid is searched, and a grid corresponding to the local maximum value is used as a target grid.

[0218] Optionally, the screening module 703 is further configured to:

[0219] Searching for a local maximum in the number of right-angle vertices in the grid;

[0220] Performing non-maximum suppression on the grid corresponding to the local maximum;

[0221] The grid corresponding to the local maximum value after non-maximum value suppression is used as the target grid.

[0222] Optionally, the connection module 704 is further configured to:

[0223] Calculating the geometric center of each right-angle vertex in the target grid respectively;

[0224] In the order in which the right-angled vertices are placed in the right-angled vertex set, each of the geometric centers is connected to form a closed right-angled polygon.

[0225] Optionally, the processing module 702 is further configured to:

[0226] Before uniformly selecting points for the projection contour, mean filtering is performed on the point sequence of the projection contour according to a preset step size or window size.

[0227] According to the various embodiments described above, it can be seen that the embodiment of the present invention obtains N points of the projection contour by uniformly selecting points from the projection contour, so that the distance from each point to the starting point of the projection contour increases in arithmetic progression, and then calculates a set of right-angled vertices based on the N points. Finally, the target grid corresponding to the local maximum value is screened out, and the geometric centers of the right-angled vertices in each target grid are connected to form a closed right-angled polygon. This technical means solves the technical problem in the prior art that the fluctuation and concavity of the projection contour affect the calculation of the right-angled polygon. The embodiment of the present invention can quickly and accurately calculate the right-angled polygon of the projection contour, greatly reducing the interference caused by factors such as image texture, lighting, color, and geometric size in subsequent matching, which helps to accurately match and locate objects; and the embodiment of the present invention has good robustness.

[0228] It should be noted that the specific implementation content of the device for processing point cloud data of the present invention has been described in detail in the method for processing point cloud data described above, so the details will not be repeated here.

[0229] Figure 8 An exemplary system architecture 800 is shown to which the method for processing point cloud data or the apparatus for processing point cloud data according to an embodiment of the present invention may be applied.

[0230] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0231] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0232] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0233] Server 805 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 801, 802, and 803. The backend management server may analyze and process received data such as item information query requests, and feed back the processing results to the terminal device.

[0234] It should be noted that the method for processing point cloud data provided in the embodiment of the present invention is generally executed by the server 805, and accordingly, the device for processing point cloud data is generally provided in the server 805. The method for processing point cloud data provided in the embodiment of the present invention can also be executed by the terminal devices 801, 802, and 803, and accordingly, the device for processing point cloud data can be provided in the terminal devices 801, 802, and 803.

[0235] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0236] Reference below Figure 9 , which shows a schematic structural diagram of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Figure 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0237] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the system 900 are also stored in the RAM 903. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0238] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0239] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0240] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0241] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer programs according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0242] The modules described in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, a processor may be described as including a projection module, a processing module, a screening module, and a connection module. The names of these modules do not, in some cases, limit the modules themselves.

[0243] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment, or may exist independently and not be incorporated into the device. The computer-readable medium carries one or more programs, and when executed by a device, the device implements the following method: obtaining point cloud data of an object, projecting the point cloud data onto a plane on which the object surface is located along a normal to the object surface to obtain a projection outline of the object; uniformly selecting points on the projection outline to obtain N points of the projection outline, such that the distance from each point to the starting point of the projection outline increases equidistantly; calculating a set of right-angled vertices based on the N points; dividing the plane on which the object surface is located into a plurality of grids, calculating the local maximum of the number of right-angled vertices falling within the plurality of grids, and thereby selecting a target grid corresponding to the local maximum; and connecting the geometric centers of the right-angled vertices in each target grid into a closed right polygon.

[0244] According to the technical solution of the embodiment of the present invention, because the projection contour is uniformly selected to obtain N points of the projection contour, the distance from each point to the starting point of the projection contour is successively increased arithmetic differences, and then a set of right-angled vertices is calculated based on the N points. Finally, the target grid corresponding to the local maximum value is screened out, and the geometric centers of the right-angled vertices in each target grid are connected to form a closed right-angled polygon. Therefore, the technical problem of the fluctuation and depression of the projection contour affecting the calculation of the right-angled polygon in the prior art is overcome. The embodiment of the present invention can quickly and accurately calculate the right-angled polygon of the projection contour, greatly reducing the interference caused by factors such as image texture, lighting, color, and geometric size in subsequent matching, which helps to accurately match and locate objects; and the embodiment of the present invention has good robustness.

[0245] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for processing point cloud data, characterized in that: include: Acquiring point cloud data of an object, and projecting the point cloud data onto a plane on which the surface of the object is located along a normal line of the surface of the object to obtain a projected outline of the object; Evenly selecting points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetic progression; calculating a set of right-angle vertices based on the N points, where N is a preset number threshold; Dividing the plane where the surface of the object is located into a plurality of grids, calculating the local maximum of the number of right-angle vertices falling in the plurality of grids, and thereby screening out the target grid corresponding to the local maximum; Connecting the geometric centers of the right-angled vertices in each target mesh into a closed right-angled polygon; The right-angle vertex set calculated based on the N points includes: For each of the N point points, calculating an angle between a first straight line passing through the point point and a point next to the point point, and a second straight line passing through the other point point and a point next to the other point point; If the absolute value of the included angle minus 90° is less than a preset angle threshold, the intersection of the first straight line and the second straight line is taken as a right-angle vertex, and the right-angle vertex is added to the right-angle vertex set.

2. The method according to claim 1, characterized in that Evenly selecting points on the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases in arithmetic progression, including: Calculating the distance from each point in the point sequence of the projection contour to the starting point of the projection contour; For each point in the point sequence, a fixed ratio is calculated according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold; If the fixed ratio is greater than or equal to zero, the coordinates of the point are calculated based on the fixed ratio, the coordinates of the point and the coordinates of the previous point of the point.

3. The method according to claim 2, characterized in that Calculating a fixed ratio according to the distance from the point to the starting point of the projection contour, the total length of the projection contour, the number of currently calculated points, and a preset number threshold, including: Dividing the distance from the point to the starting point of the projected contour by the total length of the projected contour to obtain a first quotient; Dividing the number of points currently calculated by a preset number threshold to obtain a second quotient; Subtract the first quotient from the second quotient to obtain a fixed ratio.

4. The method according to claim 2, characterized in that Calculating the coordinates of the point according to the fixed ratio, the coordinates of the point, and the coordinates of the previous point of the point, including: Multiplying the fixed ratio by the coordinates of the previous point of the point to obtain a first coordinate; Multiplying the difference of 1 minus the fixed ratio by the coordinate of the point to obtain a second coordinate; The first coordinate is added to the second coordinate to obtain the coordinates of the point.

5. The method according to claim 1, wherein Dividing the plane where the surface of the object is located into a plurality of grids, calculating the local maximum of the number of right-angle vertices falling in the plurality of grids, and thereby screening out the target grid corresponding to the local maximum, including: The plane where the surface of the object is located is evenly divided into a plurality of grids, and the number of right-angled vertices falling in each grid is calculated respectively; A local maximum value of the number of right-angle vertices in the grid is searched, and a grid corresponding to the local maximum value is used as a target grid.

6. The method according to claim 5, characterized in that Searching for a local maximum value of the number of right-angle vertices in the grid, and using a grid corresponding to the local maximum value as a target grid, including: Searching for a local maximum in the number of right-angle vertices in the grid; Performing non-maximum suppression on the grid corresponding to the local maximum; The grid corresponding to the local maximum value after non-maximum value suppression is used as the target grid.

7. The method according to claim 1, characterized in that Connecting the geometric centers of the right-angled vertices in each target mesh into a closed right-angled polygon, including: Calculating the geometric center of each right-angle vertex in the target grid respectively; In the order in which the right-angled vertices are placed in the right-angled vertex set, each of the geometric centers is connected to form a closed right-angled polygon.

8. The method according to claim 1, characterized in that Before evenly selecting points for the projection contour, the method further includes: The projection contour is subjected to mean filtering processing according to a preset step size or window size.

9. A device for processing point cloud data, characterized in that: include: a projection module, configured to acquire point cloud data of an object and project the point cloud data onto a plane on which the surface of the object is located along a normal line of the surface of the object to obtain a projection outline of the object; a processing module configured to uniformly select points from the projection contour to obtain N points of the projection contour, such that the distance from each point to the starting point of the projection contour increases arithmetically; and calculate a set of right-angle vertices based on the N points, wherein N is a preset number threshold; a screening module, configured to divide the plane on which the surface of the object is located into a plurality of grids, calculate the local maximum of the number of right-angled vertices falling in the plurality of grids, and thereby screen out a target grid corresponding to the local maximum; a connection module, configured to connect the geometric centers of the right-angled vertices in each target grid into a closed right-angled polygon; The processing module is further configured to calculate, for each of the N point points, an angle between a first straight line passing through the point and a point next to the point point, and a second straight line passing through another point point and a point next to the other point point; and if the absolute value of the angle minus 90° is less than a preset angle threshold, determine the intersection of the first straight line and the second straight line as a right-angle vertex, and add the right-angle vertex to a right-angle vertex set.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images