Coverage path generation method and apparatus, electronic device, and storage medium

Abstract

The application relates to a coverage path generation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a space point cloud of an object to be cleaned; determining a cutting plane of a point cloud region where the space point cloud is located; projecting the space point cloud to the cutting plane to obtain a projection point of the space point cloud on the cutting plane; determining a plane coverage path of the projection point on the cutting plane; and determining a space coverage path of the object to be cleaned according to the plane coverage path. The method has strong practicability.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, apparatus, electronic device, and storage medium for generating coverage paths. Background Technology

[0002] With the development of robotics technology, cleaning robots have emerged. Cleaning robots have the function of coverage path planning. Traditional coverage path planning is usually applicable to two-dimensional planes, which makes it impossible for cleaning robots to achieve good coverage on three-dimensional curved surfaces.

[0003] In existing technologies, a CAD model of the object to be cleaned can be obtained in advance. Points, lines, surfaces, or contour curves of the CAD model can then be extracted using 3D modeling software, and the cleaning robot's coverage path can be planned based on the extraction results. However, in real-world applications, the objects to be cleaned are diverse, and it may be impossible to obtain a CAD model of the object. Furthermore, performing feature extraction based on the CAD model first, and then generating a coverage path based on the extraction results, is a cumbersome process with high requirements for input model information, making it unsuitable for everyday service scenarios. Summary of the Invention

[0004] Therefore, it is necessary to provide a more practical method, apparatus, electronic device, and computer-readable storage medium for generating coverage paths to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for generating an overlay path. The method includes:

[0006] Obtain the spatial point cloud of the object to be cleaned;

[0007] Determine the cutting plane of the point cloud region where the spatial point cloud is located; the cutting plane intersects with the spatial point cloud;

[0008] The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane;

[0009] Determine the planar coverage path of the projection point on the cutting plane;

[0010] Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

[0011] Secondly, this application also provides an overlay path generation apparatus. The apparatus includes:

[0012] The point cloud acquisition module is used to acquire the spatial point cloud of the object to be cleaned;

[0013] A plane determination module is used to determine the cutting plane of the point cloud region where the spatial point cloud is located; the cutting plane intersects with the spatial point cloud;

[0014] A point cloud projection module is used to project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane.

[0015] A planar path module is used to determine the planar coverage path of the projection point on the cutting plane;

[0016] The spatial path module is used to determine the spatial coverage path of the object to be cleaned based on the planar coverage path.

[0017] Thirdly, this application also provides an electronic device, characterized in that it includes: a memory and one or more processors;

[0018] The memory is used to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors perform the following operations:

[0020] Obtain the spatial point cloud of the object to be cleaned;

[0021] Determine the cutting plane of the point cloud region where the spatial point cloud is located; the cutting plane intersects with the spatial point cloud;

[0022] The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane;

[0023] Determine the planar coverage path of the projection point on the cutting plane;

[0024] Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

[0025] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0026] Obtain the spatial point cloud of the object to be cleaned;

[0027] Determine the cutting plane of the point cloud region where the spatial point cloud is located; the cutting plane intersects with the spatial point cloud;

[0028] The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane;

[0029] Determine the planar coverage path of the projection point on the cutting plane;

[0030] Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

[0031] The aforementioned coverage path generation method, apparatus, electronic device, and storage medium first acquire the spatial point cloud of the object to be cleaned, then determine the cutting plane of the point cloud region, then project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane, and finally determine the planar coverage path of the projection points on the cutting plane. Based on the planar coverage path, the spatial coverage path of the cleaning robot for the object to be cleaned is determined. Even when the object to be cleaned is a curved surface, by projecting the spatial point cloud of the object to be cleaned onto the cutting plane and planning the coverage path for the projection points on the cutting plane, the coverage path planning problem of the three-dimensional curved surface is transformed into the coverage path planning problem of the two-dimensional plane. It eliminates the need to generate the coverage path based on the CAD model, reduces the complexity of coverage path planning, has strong practicality, and is suitable for applications in daily service scenarios. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the coverage path generation method for traditional cleaning robots;

[0033] Figure 2 This is a flowchart illustrating a method for generating an overlay path provided in Embodiment 1 of this application;

[0034] Figure 3 This is a schematic diagram of the construction of the cutting plane provided in Embodiment 1 of this application;

[0035] Figure 4 This is a schematic diagram of the projection point acquisition provided in Embodiment 1 of this application;

[0036] Figure 5 This is a schematic diagram of the projection point clustering and sorting provided in Embodiment 1 of this application;

[0037] Figure 6 This is a schematic diagram of interpolation and attitude determination provided in Embodiment 1 of this application;

[0038] Figure 7 This is a flowchart illustrating a method for generating an overlay path according to Embodiment 2 of this application;

[0039] Figure 8 This is a schematic diagram of the coverage path generation provided in Embodiment 2 of this application;

[0040] Figure 9 This is a flowchart illustrating a method for generating a robot's 3D coverage path in a service scenario, provided in Embodiment 2 of this application.

[0041] Figure 10 This is a structural block diagram of a coverage path generation device provided in Embodiment 3 of this application;

[0042] Figure 11 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] Traditional cleaning robots are typically used to clean commercial public areas or residential floors. Figure 1 A schematic diagram of the coverage path generation method for traditional cleaning robots is provided, based on... Figure 1 Coverage path generation methods can include the "ox-plowing" method and the trapezoidal decomposition method. These methods share two key characteristics: firstly, the work surface is a two-dimensional plane, and the work area and internal obstacles are typically treated as polygons; secondly, to obtain the coverage path, only the boundary contours of the workspace or obstacles need to be known, and a uniformly distributed coverage path is generated in each region by partitioning the area within the contours. However, for cleaning scenarios where the work surface is a three-dimensional curved surface, such as cleaning a car windshield, knowing only the outline of the object to be cleaned is insufficient; the shape of the surface is also required. Therefore, the ox-plowing and trapezoidal decomposition methods are not suitable.

[0045] Cleaning 3D curved surfaces is similar to traditional industrial applications such as spraying or grinding, and the methods for generating coverage paths used in spraying or grinding can be borrowed. In industrial applications, the CAD models of the parts being sprayed or ground are usually known. 3D modeling software can be used to extract points, lines, surfaces, or contour curves from the known CAD model, and then a coverage path can be generated on a specific surface. Finally, the spraying or grinding task is completed using the pose of an offline taught robot. However, in cleaning scenarios, the objects being cleaned are diverse, and CAD models of the objects are usually unavailable. Moreover, the method of extracting and editing feature curves from the CAD model to generate a coverage path is computationally intensive, cumbersome, and unsuitable for complex and ever-changing daily cleaning service scenarios. Therefore, there is currently no effective solution to the problem of generating coverage paths for 3D curved surfaces in cleaning service scenarios.

[0046] The coverage path generation method provided in this application can be applied to a terminal or a server. The terminal can be, but is not limited to, various cleaning robots, personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0047] Example 1

[0048] Figure 2 This is a flowchart illustrating a method for generating an overlay path according to Embodiment 1 of this application. For details, please refer to... Figure 2 The coverage path generation method of Embodiment 1 of this application specifically includes:

[0049] Step S110: Obtain the spatial point cloud of the object to be cleaned.

[0050] The object to be cleaned can be a curved surface, including but not limited to curved handwashing basin surfaces, curved building surfaces, and car windshields.

[0051] Among them, a spatial point cloud can be a set of three-dimensional coordinates of spatial sampling points.

[0052] In practice, the cleaning robot can be equipped with a depth sensor. This sensor can collect the spatial point cloud of the object to be cleaned and transmit it to the robot. Alternatively, the depth sensor can also collect the spatial point cloud of a target area containing the object to be cleaned and transmit this data to the robot. The robot can then extract the spatial point cloud of the object to be cleaned from the target area's point cloud. To overcome issues such as noise and incomplete information in the target area's point cloud, the cleaning robot can also acquire multiple frames of the target area's point cloud, perform 3D reconstruction on these frames, obtain the reconstructed point cloud of the target area, and extract the spatial point cloud of the object to be cleaned from this reconstructed point cloud.

[0053] For example, when cleaning a car windshield, the cleaning robot can directly acquire the point cloud of the windshield's outer surface using a depth sensor, and it can also acquire the point cloud of the car's outer surface. By using background removal and / or utilizing normal vector information, it can segment the point cloud of the windshield's outer surface from the point cloud of the car's outer surface. If the point cloud acquired by the depth sensor contains noise or incomplete information, it can perform 3D reconstruction. 3D reconstruction can fuse multiple frames of point cloud information to obtain a more accurate point cloud, and it can also filter and smooth noise in the point cloud, further improving its accuracy.

[0054] Step S120: Determine the cutting plane of the point cloud region where the spatial point cloud is located.

[0055] The point cloud region can be a spatial region that contains all spatial point clouds.

[0056] In practical implementation, either a spatial region containing the entire spatial point cloud or the smallest spatial region containing the entire spatial point cloud can be selected as the point cloud region. The cleaning robot can construct one or more two-dimensional cutting planes within the point cloud region. When constructing multiple cutting planes, they can be parallel to each other, and the distance between adjacent cutting planes can be a fixed value.

[0057] Figure 3 A schematic diagram of the construction of the cutting plane is provided. According to... Figure 3 The cutting plane can be described by a coordinate system fixed to it. This coordinate system can be used to represent the position and orientation of the cutting plane. The direction of the Y-axis of the world coordinate system can be used as the direction of the normal vector of the cutting plane to construct the normal vector. normal vector The direction is used as the direction of the Z-axis of the cutting plane coordinate system to construct an auxiliary vector. normal vector With auxiliary vector The cross product yields the Y-axis of the cutting plane coordinate system. The cross product of the Y-axis and Z-axis yields the X-axis of the cutting plane coordinate system. To ensure the cutting plane uniformly covers the point cloud, the extreme coordinates P in the point cloud can be obtained. min and P max The corresponding vector representation can be Among them, P min The value in each coordinate dimension can be the minimum value among the corresponding coordinate values ​​of the entire point cloud, P. max The values ​​in each coordinate dimension can be the maximum values ​​among the corresponding coordinate values ​​of the entire point cloud. For example, {(x k ,y k ,z k The point cloud (k = 1, 2, ..., K) represents all points on the outer surface of the car windshield, with extreme coordinates P1, P2, ..., K2 respectively. min =(x min ,y min ,z min ) and P max =(x max ,y max ,z max ), where x min =min{x k}, y min =min{y k}, z min =min{z k}, x max =max{x k}, y max =max{y k}, z max =max{z k}. If the cutting step size is set to t, then the number of cutting planes can be...

[0058]

[0059] The origin of each cutting plane can be...

[0060]

[0061] Where i represents the i-th cutting plane. Based on the origin C... i With the coordinate system X, Y, Z, each cutting plane can be determined.

[0062] Step S130: Project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane.

[0063] In practice, for each spatial point cloud, the cleaning robot can calculate its distance to each cutting plane. If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, it can be determined that the spatial point cloud is near that cutting plane. The spatial point cloud is then projected onto the cutting plane to obtain the projection point of the spatial point cloud on the cutting plane. The cleaning robot can also select spatial point clouds within a preset distance range from each cutting plane and project the selected spatial point clouds onto the cutting plane.

[0064] Figure 4 A schematic diagram of projection point acquisition is provided. Based on... Figure 4 This allows projection of points within a distance δ from the cutting plane in the point cloud onto the cutting plane. The normal vector of the cutting plane is known to be... Assume the coordinates of point M in the point cloud are (x... M y M , z M Define vectors From the origin C of the cutting plane i The vector pointing to point M, then The distance from point M to the cutting plane is Determine if d is less than δ. If d is less than δ, then M can be projected onto the cutting plane. The two-dimensional coordinates Q of the projection point on the cutting plane can be... in, These are the unit vectors of the X and Y axes of the cutting plane coordinate system, respectively. Otherwise, if d is not less than δ, M cannot be projected onto the cutting plane. In this case, other cutting planes for projection can be found for M.

[0065] Step S140: Determine the planar coverage path of the projection point on the cutting plane.

[0066] In practice, for each cutting plane, due to incomplete point cloud information or holes in the object being cleaned, the projection points are discontinuous. The cleaning robot can cluster the projection points on the cutting plane to obtain at least one cluster, with each cluster containing several discrete projection points. For each cluster on the cutting plane, the cleaning robot can sort the discrete projection points within the cluster to obtain the planar coverage path of the discrete projection points on the cutting plane.

[0067] Step S150: Determine the spatial coverage path of the object to be cleaned based on the planar coverage path.

[0068] In practice, the two-dimensional coordinates of each discrete projection point on the planar coverage path can be transformed to the world coordinate system to obtain a three-dimensional discrete path. The coordinates of each discrete projection point on the three-dimensional discrete path are represented by three-dimensional coordinates. By fitting the three-dimensional discrete path using the three-dimensional coordinates of the discrete points, the mapping relationship between the three-dimensional coordinates and the three-dimensional discrete path can be obtained. Based on this mapping relationship, the three-dimensional discrete path can be interpolated to obtain a smooth three-dimensional discrete path, which can be used as the spatial coverage path of the cleaning robot on the object to be cleaned.

[0069] It should be noted that after determining the spatial coverage path, the motion posture of the cleaning robot as it moves along the spatial coverage path on the object to be cleaned can be further determined. Specifically, when it is necessary to determine the motion posture of the cleaning robot at point P on the coverage path, the normal vector of point P on the coverage path can be determined first by using the least squares method. The normal vector is then used as the Z-axis, and the tangent at point P is used as the X-axis. Finally, the cross product of the Z-axis and the X-axis can be used to obtain the Y-axis. Based on the coordinate system X, Y, and Z of point P, the motion posture of the cleaning robot at point P can be determined. For example, after determining the coordinate axis direction, the posture of the cleaning robot can be set so that the top face is facing the same direction as the Z-axis, the front face is facing the same direction as the X-axis, and the left side face is facing the same direction as the Y-axis.

[0070] It should also be noted that the cleaning robot can plan a coverage path independently for each cluster on each cutting plane. If a break in the coverage path occurs between adjacent clusters, the cleaning robot can generate an alarm message to prompt the user for instructions, or generate a request command to send to the server, requesting instructions from the server. After receiving the instructions, the cleaning robot can perform corresponding operations. The instructions can be either to move the cleaning robot to an adjacent cluster, where the robot plans a spatial coverage path and connects the previous spatial coverage path with the current spatial coverage path to obtain a complete cleaning path; or to move the cleaning robot to an adjacent cutting plane, where the robot plans a spatial coverage path and connects the spatial coverage path on the previous plane with the current spatial coverage path to obtain a complete cleaning path.

[0071] Figure 5 A schematic diagram of projection point clustering and sorting is provided. Based on... Figure 5Due to factors such as glare and limited viewing angle, the point cloud collected by the depth sensor may have local gaps. When projected onto the cutting plane, this results in a truncation of the projected point cloud band. Since it's impossible to predict whether this truncation is due to missing data or the user's desire to avoid cleaning up the truncated area, a conservative strategy can be adopted: segmenting the projected point set on the cutting plane. Specifically, a density-based spatial clustering algorithm (DBSCAN) can be used to cluster the projected points. After clustering, the projected points of each cluster can be sorted to obtain the discrete paths corresponding to each cluster. Furthermore, the cutting plane containing the projected points can be meshed, and the mesh size can be adjusted so that each mesh contains at most one projected point. For meshes containing projected points, the coordinates of the projected points are used as the mesh coordinates; for meshes not containing projected points, the mesh coordinates are set to a preset value (∞, ∞). The search area for each search can be a 3×3 mesh region (8-neighborhood) of the current mesh's neighborhood. During the search process, the grid with the smallest x-coordinate or y-coordinate among all grids can be used as the starting grid. The nearest projection point is then searched within the 8-neighborhood of the starting grid. If no projection point is found, the search area can be expanded, or if a projection point truncation occurs, the sorting of projection points for the current cluster can be terminated, and the sorting of projection points for the next cluster can continue. Otherwise, if a projection point is found, the grid containing that projection point can be used as the second grid. The first direction vector between the starting grid and the second grid is calculated, and then a projection point is searched within the 8-neighborhood of the second grid. If no projection point is found, the search area can be expanded, or if a projection point truncation occurs, the sorting of projection points for the current cluster can be terminated, and the sorting of projection points for the next cluster can continue. The projection point truncation phenomenon terminates the sorting of projection points for the current cluster and continues to the next cluster. Otherwise, if a projection point is found, the grid containing that projection point can be used as the third grid. If two or more projection points are found, the second direction vector between the grid containing each projection point and the second grid can be calculated to obtain the angle between the second direction vector and the first direction vector. The grid containing the projection point with the smallest angle can be used as the third grid. Then, the second grid can be marked as visited, and the search for projection points in the 8-neighborhood of the third grid can continue. This process is repeated until all grids have been visited. Connecting the projection points corresponding to the starting grid, the first grid, the second grid, and so on, yields a discrete path on the current cutting plane.

[0072] To obtain the covering path on the curved surface, the two-dimensional coordinates of the projection point can be transformed to the world coordinate system to obtain the three-dimensional coordinates of the projection point. Specifically, we can first determine the projection point Q relative to the origin C of the cutting plane. i Direction vector Given a vector length d, let the coordinates of the projection point Q be (p, q), and the unit vectors of the coordinate system's X and Y axes be respectively... Then there is

[0073]

[0074] Then the three-dimensional coordinates of the projection point Q are:

[0075] Figure 6 A schematic diagram of interpolation and attitude determination is provided. Based on... Figure 6 After obtaining a two-dimensional discrete path on the cutting plane, the two-dimensional coordinates of each projection point on the discrete path are transformed into three-dimensional coordinates to obtain a three-dimensional discrete path. To facilitate the cleaning robot's execution of this path, the three-dimensional discrete path can be fitted and interpolated to obtain a smooth robot coverage path. Specifically, B-spline fitting or other regression methods can be used. Taking B-spline fitting as an example, to ensure that the fitted curve is second-order continuous, a third-order spline curve can be used. The specific model can be...

[0076]

[0077] Where N is the spline basis function, P is the spline control point, and C is the fitting point, i.e. Figure 5 The points on the discrete path obtained in the process. Given the set of fitted point samples and spline basis functions, and with N and C known, a system of equations can be constructed to solve for P. Substituting the solved P into the formula... The expression for the fitted curve can be obtained. Determine the interpolation position u, and substitute u into the formula. The corresponding interpolation points can be obtained. Connecting all the interpolation points yields a smooth covering path.

[0078] Furthermore, during the cleaning robot's operation, not only is position information needed, but also attitude information. The robot's attitude can be calculated at each interpolation point. Specifically, the normal vector at each interpolation point can be determined using the least squares method. Taking the direction of the normal vector as the direction of the Z-axis, and with the X-axis tangent to the fitted curve, the direction vector of the X-axis can be... The cross product of the unit vector along the Z-axis and the unit vector along the X-axis yields the direction of the Y-axis.

[0079] The aforementioned coverage path generation method first obtains the spatial point cloud of the object to be cleaned, then determines the cutting plane of the point cloud region, projects the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane, and finally determines the planar coverage path of the projection points on the cutting plane. Based on the planar coverage path, the spatial coverage path of the cleaning robot for the object to be cleaned is determined. This method can be used when the object to be cleaned is a curved surface. By projecting the spatial point cloud of the object to be cleaned onto the cutting plane and planning the coverage path for the projection points on the cutting plane, the coverage path planning problem of the three-dimensional curved surface is transformed into the coverage path planning problem of the two-dimensional plane. It eliminates the need to generate coverage paths based on CAD models, reduces the complexity of coverage path planning, has strong practicality, and is suitable for applications in daily service scenarios.

[0080] Example 2

[0081] Figure 7 This is a flowchart illustrating a method for generating an overlay path according to Embodiment 2 of this application. For details, please refer to... Figure 7 The coverage path generation method in Embodiment 2 of this application specifically includes:

[0082] Step S201: Obtain the spatial point cloud of the object to be cleaned.

[0083] In practice, the cleaning robot can acquire spatial point clouds collected by the depth sensor and use the spatial point clouds collected by the depth sensor as the spatial point clouds of the object to be cleaned.

[0084] In this embodiment, by acquiring the spatial point cloud of the object to be cleaned, the cleaning robot can model the object to be cleaned based on the spatial point cloud, and then determine the coverage path.

[0085] Optionally, step S201 includes:

[0086] Obtain the original depth point cloud information of the target area; the target area contains the object to be cleaned; perform 3D reconstruction based on the original depth point cloud information to obtain the target spatial point cloud corresponding to the original depth point cloud information; segment the target spatial point cloud to obtain the spatial point cloud of the object to be cleaned.

[0087] The target area can be a spatial area that contains the object to be cleaned and is larger than the area where the object to be cleaned is located.

[0088] The original depth point cloud information can be the original point cloud collected by a depth sensor. The target space point cloud can be the point cloud after 3D reconstruction of the original point cloud.

[0089] In practice, a depth sensor can collect raw depth point cloud information for a target area containing the object to be cleaned, and transmit this information to a cleaning robot. The cleaning robot can then perform 3D reconstruction based on the raw depth point cloud information of the target area to obtain the target spatial point cloud. This 3D reconstruction may include denoising, filtering, and / or smoothing operations on the raw depth point cloud information, and may also involve fusing multiple frames of raw depth point cloud information. After obtaining the target spatial point cloud, the cleaning robot can segment the target spatial point cloud using methods such as background removal and / or utilizing spatial normal vector information to obtain the spatial point cloud of the object to be cleaned.

[0090] In this embodiment, by acquiring the original depth point cloud information of the target area and performing three-dimensional reconstruction based on the original depth point cloud information, the target spatial point cloud corresponding to the original depth point cloud information can be obtained. This can solve the problems of noise and incomplete information in the original depth point cloud information, making the obtained target spatial point cloud more consistent with the actual shape of the object to be cleaned. By segmenting the target spatial point cloud, the spatial point cloud of the object to be cleaned can be obtained, making the obtained spatial point cloud of the object to be cleaned more accurate.

[0091] Step S202: Determine the cutting plane of the point cloud region where the spatial point cloud is located.

[0092] In practice, one or more two-dimensional cutting planes can be defined in the point cloud region. When multiple cutting planes are constructed, they can be parallel to each other, and the distance between adjacent cutting planes can be a fixed value.

[0093] In this embodiment, by determining the cutting plane of the point cloud region where the spatial point cloud is located, the three-dimensional point cloud can be projected onto the cutting plane to obtain two-dimensional projection points. Then, path planning is performed on the two-dimensional projection points, which reduces the complexity of coverage path planning.

[0094] Optionally, step S202 includes:

[0095] Determine the origin of the cutting plane and the first coordinate axis associated with the origin; perform a cross product between the first coordinate axis and an auxiliary vector to obtain the second coordinate axis; the angle between the auxiliary vector and the first coordinate axis meets a first preset condition; perform a cross product between the first coordinate axis and the second coordinate axis to obtain the third coordinate axis; determine the cutting plane based on the origin, the first coordinate axis, the second coordinate axis, and the third coordinate axis.

[0096] The first coordinate axis can be the Z-axis of the cutting plane coordinate system, the second coordinate axis can be the Y-axis of the cutting plane coordinate system, and the third coordinate axis can be the X-axis of the cutting plane coordinate system.

[0097] The first preset condition can be that the angle between the auxiliary vector and the first coordinate axis is an acute angle.

[0098] It should be noted that when there are multiple cutting planes, the origin, first coordinate axis, second coordinate axis, and third coordinate axis of each cutting plane can be determined separately. A coordinate system can be established based on the determined origin, first coordinate axis, second coordinate axis, and third coordinate axis, and each cutting plane can be determined based on the established coordinate system.

[0099] In specific implementation, according to Figure 3 The cutting plane can be described by a coordinate system fixed to it. This coordinate system can be used to represent the position and orientation of the cutting plane. The direction of the Y-axis of the world coordinate system can be used as the direction of the normal vector of the cutting plane to construct the normal vector. normal vector The direction is used as the direction of the Z-axis of the cutting plane coordinate system to construct an auxiliary vector. When the angle between the auxiliary vector and the Z-axis is less than 90 degrees, the normal vector can be... With auxiliary vector The cross product yields the Y-axis of the cutting plane coordinate system. The cross product of the Y-axis and Z-axis yields the X-axis of the cutting plane coordinate system. To ensure the cutting plane uniformly covers the point cloud, the extreme coordinates P in the point cloud can be obtained. min and P max The corresponding vector representation can be Among them, P min The value in each coordinate dimension can be the minimum value among the corresponding coordinate values ​​of the entire point cloud, P. max The values ​​in each coordinate dimension can be the maximum values ​​among the corresponding coordinate values ​​of the entire point cloud. For example, {(x k ,y k ,z k The set of points (k = 1, 2, ..., K) represents the entire spatial point cloud, with extreme coordinates P1, P2, ..., K2. min =(x min ,y min ,z min ) and P max =(x max ,y max ,z max ), where x min =min{x k}, y min =min{y k}, z min =min{z k}, x max =max{x k}, y max =max{yk}, z max =max{z k}. If the cutting step size is set to t, then the number of cutting planes can be...

[0100]

[0101] The origin of each cutting plane can be...

[0102]

[0103] Where i represents the i-th cutting plane. Based on the origin C... i With the coordinate system X, Y, Z, each cutting plane can be determined.

[0104] In this embodiment, by determining the origin of the cutting plane and the first coordinate axis associated with the origin, the first coordinate axis is cross-producted with an auxiliary vector to obtain the second coordinate axis, and the first coordinate axis is cross-producted with the second coordinate axis to obtain the third coordinate axis. Based on the origin, the first coordinate axis, the second coordinate axis, and the third coordinate axis, the cutting plane is determined, and a coordinate system for the cutting plane can be established. Then, the cutting plane is determined based on the established coordinate system, thereby automatically determining the cutting plane and improving the efficiency of cutting plane determination.

[0105] Optionally, determining the origin of the cutting plane includes:

[0106] Determine the coordinates of the first and second point clouds in the spatial point cloud; the coordinates of the first point cloud meet the second preset condition, and the coordinates of the second point cloud meet the third preset condition; determine the point cloud coordinate vector; the point cloud coordinate vector corresponds to the coordinates of the first and second point clouds; divide the point cloud coordinate vector by the preset cutting step size to obtain the number of cutting planes; determine the origin of the cutting planes based on the coordinates of the first point cloud, the cutting step size, and the number of cutting planes.

[0107] The first point cloud coordinates can be the coordinates of one spatial point cloud, and the second point cloud coordinates can be the coordinates of another spatial point cloud.

[0108] The second preset condition can be that the value of the first point cloud coordinates in each coordinate dimension is the minimum value among the corresponding coordinate values ​​of all point clouds. The third preset condition can be that the value of the second point cloud coordinates in each coordinate dimension is the maximum value among the corresponding coordinate values ​​of all point clouds.

[0109] In the specific implementation, the coordinates P of the first point cloud in the spatial point cloud can be obtained. min Second point cloud coordinates P max , where P min The value in each coordinate dimension can be the minimum value among the corresponding coordinate values ​​of the entire spatial point cloud, P.max The values ​​in each coordinate dimension can be the maximum values ​​among the corresponding coordinate values ​​of the entire spatial point cloud. For example, it can be done using {(x k ,y k ,z k The set of points (k = 1, 2, ..., K) represents the entire spatial point cloud, with extreme coordinates P1, P2, ..., K2. min =(x min ,y min ,z min ) and P max =(x max ,y max ,z max ), where x min =min{x k}, y min =min{y k}, z min =min{z k}, x max =max{x k}, y max =max{y k}, z max =max{z k The vectors corresponding to the first and second point cloud coordinates. For point cloud coordinate vectors, set the cutting step size to t, and according to... And t, we can get the number of cutting planes as

[0110]

[0111] According to P min From t and N, we can obtain the origin of the cutting plane as...

[0112]

[0113] Where i represents the i-th cutting plane.

[0114] In this embodiment, by determining the coordinates of the first point cloud and the second point cloud in the spatial point cloud, the point cloud coordinate vector is determined. The point cloud coordinate vector is divided by the preset cutting step size to obtain the number of cutting planes. Based on the first point cloud coordinates, the cutting step size and the number of cutting planes, the origin of the cutting plane is determined, which allows the cutting plane to uniformly cover all spatial point clouds, thereby enabling each spatial point cloud to be projected onto at least one cutting plane.

[0115] Step S203: Project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane.

[0116] In practice, for each spatial point cloud, the distance between it and each cutting plane can be calculated. If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the spatial point cloud can be projected onto the cutting plane to obtain the projection point of the spatial point cloud on the cutting plane. Alternatively, for each cutting plane, spatial point clouds within a preset distance range from the cutting plane can be selected, and the selected spatial point clouds can be projected onto the cutting plane.

[0117] In this embodiment, by projecting the spatial point cloud onto the cutting plane, the projection points of the spatial point cloud on the cutting plane are obtained, which can transform the three-dimensional spatial point cloud into two-dimensional projection points, reducing the complexity of coverage path planning.

[0118] Optionally, step S203 includes:

[0119] If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the spatial point cloud is projected onto the cutting plane to obtain the projection point.

[0120] In practice, the distance between the spatial point cloud and the cutting plane can be counted. If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane.

[0121] In this embodiment, by projecting the spatial point cloud onto the cutting plane to obtain projection points when the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the three-dimensional spatial point cloud can be transformed into two-dimensional projection points, reducing the complexity of coverage path planning.

[0122] Step S204: Cluster the projection points on the cutting plane to obtain clusters of projection points;

[0123] Step S205: Sort the projection points within the cluster to obtain the planar coverage path of the projection points on the cutting plane.

[0124] In specific implementation, according to Figure 5A density-based clustering algorithm can be used to cluster the projection points on the cutting plane. After clustering, the projection points of each cluster are sorted to obtain the discrete paths corresponding to each cluster. Specifically, the cutting plane containing projection points can be meshed, and the mesh size can be adjusted so that each mesh contains at most one projection point. For meshes containing projection points, the coordinates of the projection points are used as the mesh coordinates; for meshes not containing projection points, the mesh coordinates are set to a preset value (∞, ∞). The search area for each search can be a 3×3 mesh area (8-neighborhood) of the current mesh's neighborhood. During the search process, the grid with the smallest x-coordinate or y-coordinate among all grids can be used as the starting grid. The nearest projection point is then searched within the 8-neighborhood of the starting grid. If no projection point is found, the search area can be expanded, or if a projection point truncation occurs, the sorting of projection points for the current cluster can be terminated, and the sorting of projection points for the next cluster can continue. Otherwise, if a projection point is found, the grid containing that projection point can be used as the second grid. The first direction vector between the starting grid and the second grid is calculated, and then a projection point is searched within the 8-neighborhood of the second grid. If no projection point is found, the search area can be expanded, or if a projection point truncation occurs, the sorting of projection points for the current cluster can be terminated, and the sorting of projection points for the next cluster can continue. The projection point truncation phenomenon terminates the sorting of projection points for the current cluster and continues to the next cluster. Otherwise, if a projection point is found, the grid containing that projection point can be used as the third grid. If two or more projection points are found, the second direction vector between the grid containing each projection point and the second grid can be calculated to obtain the angle between the second direction vector and the first direction vector. The grid containing the projection point with the smallest angle can be used as the third grid. Then, the second grid can be marked as visited, and the search for projection points in the 8-neighborhood of the third grid can continue. This process is repeated until all grids have been visited. Connecting the projection points corresponding to the starting grid, the first grid, the second grid, and so on, yields a discrete path on the current cutting plane.

[0125] In this embodiment, by clustering the projection points on the cutting plane, clusters of projection points are obtained. In the case of local missing points in the spatial point cloud, the missing parts are not automatically processed to avoid erroneous operations. The projection points in the clusters are sorted to obtain the coverage path of the projection points on the cutting plane. The coverage path can be planned for the projection points in the clusters, thereby realizing the coverage path planning for the cleaning robot.

[0126] Optionally, step S205 includes:

[0127] Based on the search grid on the cutting plane, determine the first projection point, the second projection point, and the first direction vector among the projection points within the cluster; the distance between the second projection point and the first projection point meets the fourth preset condition; the first direction vector is the direction vector between the first projection point and the second projection point; based on the search grid of the neighborhood of the second projection point, determine the third projection point and the second direction vector; the second direction vector is the direction vector between the second projection point and the third projection point; the angle between the second direction vector and the first direction vector meets the fifth preset condition; determine the number of grids to be searched in the search grid, take the third projection point as the new second projection point, take the second direction vector as the new first direction vector, and return to the step of determining the third projection point and the second direction vector based on the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets the sixth preset condition.

[0128] The fourth preset condition can be that, among all projection points, the second projection point is closest to the first projection point.

[0129] The fifth preset condition can be that, among all the direction vectors associated with the second projection point, the angle between the second direction vector and the first direction vector is the smallest.

[0130] The sixth preset condition can be that the number of grids to be searched is 0.

[0131] In the specific implementation, the cutting plane can be meshed to obtain a search grid. During the search process, the point with the smallest x-coordinate or y-coordinate among all projection points can be taken as the first projection point. Then, the nearest projection point to the first projection point is searched in the neighborhood of the grid containing the first projection point to obtain the second projection point. The direction vector between the first projection point and the second projection point is taken as the first direction vector, and a search mark is added to the grid containing the first projection point and the second projection point.

[0132] Then, search for projection points in the neighborhood of the grid where the second projection point is located. If two or more projection points are found, calculate the direction vector between each projection point and the second projection point, and calculate the angle between the direction vector and the first direction vector. Select the smallest angle, determine the projection point corresponding to the smallest angle as the third projection point, determine the direction vector corresponding to the smallest angle as the second direction vector, and add a search mark to the grid where the third projection point is located.

[0133] By removing grids with searched markers from the search grid, the number of grids to be searched is calculated. If the number of grids to be searched is not 0, the third projection point can be used as the new second projection point, and the second direction vector can be used as the new first direction vector. The above process is repeated until the number of grids to be searched in the search grid is 0, and the loop ends.

[0134] By sorting all the projection points within a cluster according to their determined order, a planar covering path for each projection point within the cluster can be obtained. Similarly, by sorting all the clusters on the cutting plane, all planar covering paths on the cutting plane can be obtained.

[0135] In this embodiment, based on the search grid on the cutting plane, a first projection point, a second projection point, and a first direction vector are determined among the projection points within the cluster. Based on the search grid of the neighborhood of the second projection point, a third projection point and a second direction vector are determined. The number of grids to be searched in the search grid is determined. The third projection point is used as the new second projection point, and the second direction vector is used as the new first direction vector. The process returns to the step of determining the third projection point and the second direction vector based on the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets the sixth preset condition. This allows for a comprehensive search of the projection points on the cutting plane, avoiding omissions of cleaning points. By ensuring that the distance between the second projection point and the first projection point meets the fourth preset condition, and the angle between the second direction vector and the first direction vector meets the fifth preset condition, and through iterative iteration, the cleaning robot travels the shortest distance and requires the least amount of turning when moving from one cleaning point to the next on the planned path, thus saving robot energy.

[0136] Step S206: Perform coordinate transformation on the projection points on the plane covering path to obtain the coordinate transformed projection points;

[0137] Step S207: Fit the projection points after coordinate transformation to obtain the spatial coverage path of the object to be cleaned;

[0138] Step S208: Determine the target position on the spatial coverage path of the object to be cleaned; determine the first coordinate axis and the second coordinate axis of the cleaning robot at the target position based on the normal vector and the tangent direction vector of the target position, respectively; perform a cross product of the first coordinate axis and the second coordinate axis to obtain the third coordinate axis of the cleaning robot at the target position; determine the posture of the cleaning robot at the target position based on the first coordinate axis, the second coordinate axis and the third coordinate axis.

[0139] The target location can be any point on the spatial coverage path of the object to be cleaned obtained through fitting.

[0140] The first coordinate axis can be the Z-axis of the cleaning robot, the second coordinate axis can be the X-axis of the cleaning robot, and the third coordinate axis can be the Y-axis of the cleaning robot.

[0141] In the specific implementation, we can first determine the projection point Q on the plane covering path relative to the origin C of the cutting plane. i Direction vector Given a vector length d, let the coordinates of the projection point Q be (p, q), and the unit vectors of the coordinate system's X and Y axes be respectively... Then there is

[0142]

[0143] According to the direction vector Given the vector length d, the three-dimensional coordinates of the projection point Q can be obtained. Performing the above processing on each projection point on the planar coverage path yields the coordinate-transformed 3D projection points. Based on the coverage path of the projection points on the cutting plane, sorting the coordinate-transformed 3D projection points yields the coordinate-transformed planar coverage path, which can be represented by a series of 3D coordinates. Let the fitting model be...

[0144]

[0145] Where N is the spline basis function, P is the spline control point, and C is the fitted point. Given the sample set of fitted points and the spline basis function, and with N and C known, a system of equations can be constructed to solve for P. Substituting the solved P into the formula... The expression for the fitted curve can be obtained. Determine the interpolation position u, and substitute u into the formula. The corresponding interpolation points can be obtained. By connecting all the interpolation points, a smooth three-dimensional spatial coverage path can be obtained for the cleaning robot on the object to be cleaned.

[0146] Next, the orientation of the cleaning robot can be determined. Specifically, the normal vector at each interpolation point can be determined using the least squares method. The direction of the normal vector can be used as the Z-axis direction of the cleaning robot at the interpolation point. Alternatively, the tangent direction of the fitted curve at the interpolation point can be used as the X-axis direction. The X-axis direction vector can be... The Y-axis direction can be obtained by cross-product of the Z-axis unit vector and the X-axis unit vector. Based on the determined X, Y, and Z axes, the pose of the cleaning robot at the interpolation point can be obtained. By determining the pose of the cleaning robot at all interpolation points using the above method, all poses of the cleaning robot along the spatial coverage path can be obtained.

[0147] In this embodiment, by performing coordinate transformation on the projection points on the planar coverage path, the two-dimensional projection points can be transformed into three-dimensional coordinates to obtain three-dimensional projection points. Fitting the coordinate-transformed projection points yields the spatial coverage path of the object to be cleaned, making the spatial coverage path of the object to be cleaned smoother and facilitating robot execution. The target position is determined on the spatial coverage path of the object to be cleaned, and the posture of the cleaning robot at the target position is determined based on the normal vector and tangent direction vector of the target position. This yields all postures of the cleaning robot on the spatial coverage path, which can guide the cleaning robot's posture on the coverage path and facilitate more accurate guidance for the cleaning robot to perform its work.

[0148] To facilitate a deeper understanding of the embodiments of this application by those skilled in the art, a specific example will be used for illustration below.

[0149] Figure 8 A schematic diagram of the coverage path generation is provided. According to... Figure 8 When planning a coverage path for a cleaning robot on a curved surface, a point cloud representation model of the object to be cleaned can be obtained first. Then, by cutting the object model and projecting points near the cutting surface onto the cutting surface, a series of two-dimensional projection points are obtained. These projection points are then clustered, sorted, and interpolated to obtain the coverage path. This method relies only on point cloud input and can be post-processed by adding reconstruction algorithms to the robot's depth camera, making it highly practical. Moreover, this method applies techniques such as clustering, sorting, and B-spline fitting, which can effectively compensate for problems such as missing local point clouds, resulting in a smoother path that is beneficial for robot execution.

[0150] Figure 9 This is a flowchart illustrating a method for generating a robot's 3D coverage path in a service scenario, as provided in Embodiment 2 of this application. For details, please refer to... Figure 9 The robot 3D coverage path generation method for service scenarios in Embodiment 2 of this application specifically includes:

[0151] Step S310: Obtain the point cloud model of the object to be cleaned. The difference between service and industrial scenarios is the lack of a pre-known CAD model; however, the robot can obtain the point cloud model of the object using its own depth sensors. The original point cloud contains noise and incomplete information, therefore requiring processing using a 3D reconstruction algorithm. This algorithm can fuse multiple frames of point cloud information into a complete scene point cloud model and performs filtering and smoothing operations on the original noise, resulting in a more accurate point cloud. After obtaining the complete point cloud, the point cloud of the object to be cleaned needs to be extracted. This can be done by segmenting the target object using methods such as background removal and normal vector information. Finally, a complete and accurate point cloud model of the target object is obtained.

[0152] Step S320: Construct the cutting plane. The plane can be described by a coordinate system fixed on the plane, which means determining the origin, normal vector, and a vector on the plane. In this invention, it is assumed that the normal vector of the cutting plane is given artificially, for example, along the direction of the Y-axis of the world coordinate system, i.e., a vector. This direction is the Z-axis of the cutting plane coordinate system. To obtain the X and Y axes, an auxiliary vector is constructed. Using vectors Cross product vector The Y-axis of the cutting surface is obtained, and finally, the X-axis is obtained by cross-multiplying the Y-axis by the Z-axis. The cutting surface should uniformly cover the point cloud model; therefore, the extreme coordinates P are first obtained from the point cloud model. min and P max The corresponding vector representation is If the cutting step size is t, then the total number of cutting faces is... Origin of each cut surface i represents the i-th cutting surface.

[0153] Step S330: Obtain the projection points. Project the points in the point cloud model within a distance δ from the cutting surface onto the cutting surface. The normal vector of the cutting surface is known to be... Assuming the coordinates of point M in the point cloud model are (x, y, z), and defining vector s as the vector pointing from the origin Ci of the tangent plane to M, then... Distance of point M from the cutting surface Determine if d is less than δ; if so, project onto the cutting plane. The 2D projection coordinates Q are... These are the unit vectors of the X and Y axes of the cutting surface coordinate system, respectively.

[0154] Step S340: Clustering and Sorting. Due to factors such as reflection and limited viewing angle, the obtained point cloud model often exhibits local point cloud gaps, resulting in truncation of the 2D point cloud band when projected onto the cutting surface. Since it's impossible to predict whether this truncation is due to missing data or if the area was never intended to be cleaned, a conservative strategy is to segment the projected point cloud rather than subjectively stitching them together. To segment, it's necessary to first determine how many segments the 2D projected point cloud should be divided into and which segment each projected point belongs to. To achieve this, this invention uses the DBSCAN density-based clustering method to cluster the projected points. The advantage of using DBSCAN is that it eliminates the need to pre-specify the number of segments to be processed for the coverage path, and secondly, DBSCAN has good robustness against outliers. After clustering, the projected points of each cluster need to be sorted. The purpose of sorting is to obtain a monotonically ordered discrete path point so that the robot can execute the cleaning path in the correct order.

[0155] To obtain the discrete points of these covered paths, the 2D projection surface is first meshed. If a projection point exists within a mesh, its center is used to represent the mesh coordinates. Each time, a 3×3 mesh region surrounding the current mesh is searched. Initially, the point with the smallest coordinates among the projection points is used as the starting search point. The first search uses the nearest neighboring point as the next target, calculating the direction vectors of these two points. Then, the eight neighboring meshes of the current point are searched. If a mesh contains a projection point, the direction vectors of the current point and the mesh point are calculated. The angle between adjacent direction vectors is compared sequentially, and the mesh with the smallest angle is selected as the next target, marking the current mesh as visited. This process iterates until there are no unvisited values ​​around the mesh, at which point the iteration stops, ultimately yielding a discrete path of projection points.

[0156] To obtain the final coverage path, the 2D coordinates of the projection point need to be transformed back to the world coordinate system to obtain the 3D coordinates. First, obtain the direction vector g and vector length d of the projection point relative to the origin of the cutting surface. Following the previous notation, the coordinates of the projection point are (p, q), and the X and Y axis vectors of the coordinate system are... but Therefore, the final 3D coordinates of Q are

[0157] Step S350: Following step S340, a series of discrete points on the covered path are obtained. To obtain a smooth, robot-executable path, further fitting of these discrete points is required. B-spline fitting or other regression methods can be used. Taking B-spline fitting as an example, to ensure the fitted curve is second-order continuous, a third-order spline curve is used, and its model is as follows: Where N is the spline basis function, P is the spline control point, and C is the given set of fitting points. A system of equations can be constructed using the fitting points, and by solving for the control points P, the expression C of the spline curve can be obtained.

[0158] Step S360, having obtained the fitted model C, allows us to obtain the corresponding interpolation point P by providing the interpolation position u. Because robot cleaning requires not only positional accuracy but also the robot's pose, the corresponding pose o needs to be calculated at point P. To determine o, the normal vector of point P can be obtained using the least squares method from the point cloud model near point P. The X-axis of o is tangent to the fitted curve, and the tangent direction vector is... The Y-axis direction is the cross product of the Z-axis unit vector and the X-axis unit vector.

[0159] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0160] Example 3

[0161] Based on the same inventive concept, this application also provides a coverage path generation apparatus for implementing the coverage path generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more coverage path generation apparatus embodiments provided below can be found in the limitations of the coverage path generation method described above, and will not be repeated here.

[0162] In one embodiment, such as Figure 10 As shown, a coverage path generation device is provided, including: a point cloud acquisition module 410, a plane determination module 420, a point cloud projection module 430, a plane path module 440, and a spatial path module 450, wherein:

[0163] Point cloud acquisition module 410 is used to acquire the spatial point cloud of the object to be cleaned;

[0164] The plane determination module 420 is used to determine the cutting plane of the point cloud region where the spatial point cloud is located;

[0165] The point cloud projection module 430 is used to project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane.

[0166] Plane path module 440 is used to determine the planar coverage path of the projection point on the cutting plane;

[0167] The spatial path module 450 is used to determine the spatial coverage path of the object to be cleaned based on the planar coverage path.

[0168] In another embodiment, the point cloud acquisition module 410 is further configured to: acquire the original depth point cloud information of the target area; the target area includes the object to be cleaned; perform three-dimensional reconstruction based on the original depth point cloud information to obtain the target spatial point cloud corresponding to the original depth point cloud information; and segment the target spatial point cloud to obtain the spatial point cloud of the object to be cleaned.

[0169] In another embodiment, the plane determination module 420 described above further includes:

[0170] An origin determination module is used to determine the origin of the cutting plane and the first coordinate axis associated with the origin;

[0171] The first cross product module is used to perform a cross product between the first coordinate axis and the auxiliary vector to obtain the second coordinate axis; the angle between the auxiliary vector and the first coordinate axis meets a first preset condition;

[0172] The second cross product module is used to perform a cross product between the first coordinate axis and the second coordinate axis to obtain the third coordinate axis;

[0173] The cutting plane determination module is used to determine the cutting plane based on the origin, the first coordinate axis, the second coordinate axis, and the third coordinate axis.

[0174] In another embodiment, the origin determination module is further configured to: determine the coordinates of a first point cloud and a second point cloud in the spatial point cloud; the first point cloud coordinates meet a second preset condition, and the second point cloud coordinates meet a third preset condition; determine a point cloud coordinate vector; the point cloud coordinate vector corresponds to the first point cloud coordinates and the second point cloud coordinates; divide the point cloud coordinate vector by a preset cutting step size to obtain the number of cutting planes; and determine the origin of the cutting planes based on the first point cloud coordinates, the cutting step size, and the number of cutting planes.

[0175] In another embodiment, the point cloud projection module 430 is further configured to: project the spatial point cloud onto the cutting plane to obtain the projection point, provided that the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold.

[0176] In another embodiment, the planar path module 440 described above further includes:

[0177] A clustering module is used to cluster the projection points on the cutting plane to obtain clusters of the projection points.

[0178] The sorting module is used to sort the projection points within the cluster to obtain the planar coverage path of the projection points on the cutting plane.

[0179] In another embodiment, the sorting module is further configured to: determine a first projection point, a second projection point, and a first direction vector among the projection points within the cluster based on the search grid on the cutting plane; the distance between the second projection point and the first projection point meets a fourth preset condition; the first direction vector is the direction vector between the first projection point and the second projection point; determine a third projection point and a second direction vector based on the search grid of the neighborhood of the second projection point; the second direction vector is the direction vector between the second projection point and the third projection point; the angle between the second direction vector and the first direction vector meets a fifth preset condition; determine the number of grids to be searched in the search grid; use the third projection point as a new second projection point; use the second direction vector as a new first direction vector; and return to the step of determining the third projection point and the second direction vector based on the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets a sixth preset condition.

[0180] In another embodiment, the spatial path module 450 further includes:

[0181] The coordinate transformation module is used to perform coordinate transformation on the projection points on the plane coverage path to obtain the coordinate transformed projection points.

[0182] The path fitting module is used to fit the projection points after the coordinate transformation to obtain the spatial coverage path of the object to be cleaned.

[0183] In another embodiment, the above-described coverage path generation apparatus further includes:

[0184] The target location determination module is used to determine the target location on the spatial coverage path of the object to be cleaned;

[0185] The first attitude determination module is used to determine the first coordinate axis and the second coordinate axis of the cleaning robot at the target position based on the normal vector and the tangent direction vector of the target position, respectively.

[0186] The second attitude determination module is used to perform a cross product of the first coordinate axis and the second coordinate axis to obtain the third coordinate axis of the cleaning robot at the target position;

[0187] The target posture determination module is used to determine the posture of the cleaning robot at the target position based on the first coordinate axis, the second coordinate axis and the third coordinate axis.

[0188] The above-described overlay path generation apparatus can be used to execute the overlay path generation method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0189] Each module in the aforementioned coverage path generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0190] Example 4

[0191] Figure 11 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of this application. As shown in the figure, the electronic device includes: a processor 50, a memory 51, a touch-enabled display screen 52, an input device 53, an output device 54, and a communication device 55. The electronic device may have one or more processors 50; the figure shows one processor 50 as an example. The electronic device may have one or more memory units 51; the figure shows one memory unit 51 as an example. The processor 50, memory 51, display screen 52, input device 53, output device 54, and communication device 55 of the electronic device can be connected via a bus or other means; the figure shows a bus connection as an example. In this embodiment, the electronic device may be a computer, mobile phone, tablet, projector, or interactive smart tablet, etc. In this embodiment, an interactive smart tablet is used as an example for description.

[0192] The memory 51, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the overlay path generation method described in any embodiment of this application. The memory 51 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 51 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include memory remotely located relative to the processor 50, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0193] The display screen 52 is a touch-enabled display screen, which can be a capacitive screen, an electromagnetic screen, or an infrared screen. Generally, the display screen 52 is used to display data according to the instructions of the processor 50, and also to receive touch operations applied to the display screen 52 and send corresponding signals to the processor 50 or other devices. Optionally, when the display screen 52 is an infrared screen, it also includes an infrared touch frame, which is disposed around the perimeter of the display screen 52. This infrared touch frame can also be used to receive infrared signals and send the infrared signals to the processor 50 or other devices.

[0194] Communication device 55 is used to establish communication connections with other devices, and may be a wired communication device and / or a wireless communication device.

[0195] Input device 53 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of electronic devices. It can also be a camera for acquiring images and a sound pickup device for acquiring audio data. Output device 54 may include audio devices such as speakers. It should be noted that the specific composition of input device 53 and output device 54 can be set according to actual conditions.

[0196] The processor 50 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 51, thereby realizing the above-mentioned overlay path generation method.

[0197] Specifically, in this embodiment, when the processor 50 executes one or more programs stored in the memory 51, it performs the following operations:

[0198] Obtain the spatial point cloud of the object to be cleaned;

[0199] Determine the cutting plane of the point cloud region where the spatial point cloud is located;

[0200] The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane;

[0201] Determine the planar coverage path of the projection point on the cutting plane;

[0202] Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

[0203] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0204] Obtain the original depth point cloud information of the target area; the target area contains the object to be cleaned.

[0205] Based on the original depth point cloud information, a three-dimensional reconstruction is performed to obtain the target spatial point cloud corresponding to the original depth point cloud information;

[0206] The target spatial point cloud is segmented to obtain the spatial point cloud of the object to be cleaned.

[0207] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0208] Determine the origin of the cutting plane and the first coordinate axis associated with the origin;

[0209] The first coordinate axis is cross-productted with the auxiliary vector to obtain the second coordinate axis; the angle between the auxiliary vector and the first coordinate axis meets the first preset condition;

[0210] The third coordinate axis is obtained by cross-product of the first coordinate axis and the second coordinate axis;

[0211] The cutting plane is determined based on the origin, the first coordinate axis, the second coordinate axis, and the third coordinate axis.

[0212] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0213] Determine the coordinates of a first point cloud and a second point cloud in the spatial point cloud; the coordinates of the first point cloud meet a second preset condition, and the coordinates of the second point cloud meet a third preset condition.

[0214] Determine the point cloud coordinate vector; the point cloud coordinate vector corresponds to the first point cloud coordinate and the second point cloud coordinate.

[0215] Divide the point cloud coordinate vector by the preset cutting step size to obtain the number of cutting planes;

[0216] The origin of the cutting plane is determined based on the first point cloud coordinates, the cutting step size, and the number of cutting planes.

[0217] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0218] If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the spatial point cloud is projected onto the cutting plane to obtain the projection point.

[0219] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0220] Cluster the projection points on the cutting plane to obtain clusters of the projection points;

[0221] The projection points within the cluster are sorted to obtain the planar coverage path of the projection points on the cutting plane.

[0222] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0223] Based on the search grid on the cutting plane, a first projection point, a second projection point, and a first direction vector are determined among the projection points within the cluster; the distance between the second projection point and the first projection point meets a fourth preset condition; the first direction vector is the direction vector between the first projection point and the second projection point.

[0224] Based on the search grid of the neighborhood of the second projection point, a third projection point and a second direction vector are determined; the second direction vector is the direction vector between the second projection point and the third projection point; the angle between the second direction vector and the first direction vector meets the fifth preset condition.

[0225] The number of grids to be searched in the search grid is determined, the third projection point is used as the new second projection point, the second direction vector is used as the new first direction vector, and the process returns to the step of determining the third projection point and the second direction vector based on the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets the sixth preset condition.

[0226] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0227] Perform coordinate transformation on the projection points on the plane coverage path to obtain the coordinate-transformed projection points;

[0228] The spatial coverage path of the object to be cleaned is obtained by fitting the projection points after the coordinate transformation.

[0229] Based on the above embodiments, one or more processors 50 further implement the following operations:

[0230] Determine the target location on the spatial coverage path of the object to be cleaned;

[0231] The first and second coordinate axes of the cleaning robot at the target position are determined based on the normal vector and tangent direction vector of the target position, respectively.

[0232] The third coordinate axis of the cleaning robot at the target position is obtained by cross-product of the first and second coordinate axes.

[0233] The posture of the cleaning robot at the target location is determined based on the first coordinate axis, the second coordinate axis, and the third coordinate axis.

[0234] Example 5

[0235] Embodiment 5 of this application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an overlay path generation method, including:

[0236] Obtain the spatial point cloud of the object to be cleaned;

[0237] Determine the cutting plane of the point cloud region where the spatial point cloud is located;

[0238] The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane;

[0239] Determine the planar coverage path of the projection point on the cutting plane;

[0240] Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

[0241] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the operations of the overlay path generation method as described above, but can also execute related operations in the overlay path generation method provided in any embodiment of this application, and have corresponding functions and beneficial effects.

[0242] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0243] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0244] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating an overlay path, characterized in that, The method includes: Obtain the spatial point cloud of the object to be cleaned; Determine the cutting plane of the point cloud region where the spatial point cloud is located; The spatial point cloud is projected onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane; Cluster the projection points on the cutting plane to obtain clusters of the projection points; Based on the search grid on the cutting plane, a first projection point, a second projection point, and a first direction vector are determined among the projection points within the cluster; the distance between the second projection point and the first projection point meets a fourth preset condition; the first direction vector is the direction vector between the first projection point and the second projection point. Based on the search grid of the neighborhood of the second projection point, a third projection point and a second direction vector are determined; the second direction vector is the direction vector between the second projection point and the third projection point; the angle between the second direction vector and the first direction vector meets the fifth preset condition. The number of grids to be searched in the search grid is determined, the third projection point is used as the new second projection point, the second direction vector is used as the new first direction vector, and the process returns to the step of determining the third projection point and the second direction vector based on the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets the sixth preset condition, and the planar coverage path of the projection point on the cutting plane is obtained. Based on the planar coverage path, determine the spatial coverage path of the object to be cleaned.

2. The method according to claim 1, characterized in that, The acquisition of the spatial point cloud of the object to be cleaned includes: Obtain the original depth point cloud information of the target area; the target area contains the object to be cleaned. Based on the original depth point cloud information, a three-dimensional reconstruction is performed to obtain the target spatial point cloud corresponding to the original depth point cloud information; The target spatial point cloud is segmented to obtain the spatial point cloud of the object to be cleaned.

3. The method according to claim 1, characterized in that, The step of determining the cutting plane of the point cloud region where the spatial point cloud is located includes: Determine the origin of the cutting plane and the first coordinate axis associated with the origin; The first coordinate axis is cross-productted with the auxiliary vector to obtain the second coordinate axis; the angle between the auxiliary vector and the first coordinate axis meets the first preset condition; The third coordinate axis is obtained by cross-product of the first coordinate axis and the second coordinate axis; The cutting plane is determined based on the origin, the first coordinate axis, the second coordinate axis, and the third coordinate axis.

4. The method according to claim 3, characterized in that, Determining the origin of the cutting plane includes: Determine the coordinates of a first point cloud and a second point cloud in the spatial point cloud; the coordinates of the first point cloud meet a second preset condition, and the coordinates of the second point cloud meet a third preset condition. Determine the point cloud coordinate vector; the point cloud coordinate vector corresponds to the first point cloud coordinate and the second point cloud coordinate. Divide the point cloud coordinate vector by the preset cutting step size to obtain the number of cutting planes; The origin of the cutting plane is determined based on the first point cloud coordinates, the cutting step size, and the number of cutting planes.

5. The method according to claim 1, characterized in that, The step of projecting the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane includes: If the distance between the spatial point cloud and the cutting plane does not exceed a preset distance threshold, the spatial point cloud is projected onto the cutting plane to obtain the projection point.

6. The method according to claim 1, characterized in that, Determining the spatial coverage path of the object to be cleaned based on the planar coverage path includes: Perform coordinate transformation on the projection points on the plane coverage path to obtain the coordinate-transformed projection points; The spatial coverage path of the object to be cleaned is obtained by fitting the projection points after the coordinate transformation.

7. The method according to claim 6, characterized in that, After fitting the projected points after the coordinate transformation to obtain the spatial coverage path of the object to be cleaned, the process further includes: Determine the target location on the spatial coverage path of the object to be cleaned; The first and second coordinate axes of the cleaning robot at the target position are determined based on the normal vector and tangent direction vector of the target position, respectively. The third coordinate axis of the cleaning robot at the target position is obtained by cross-product of the first and second coordinate axes. The posture of the cleaning robot at the target location is determined based on the first coordinate axis, the second coordinate axis, and the third coordinate axis.

8. A coverage path generation device, characterized in that, The device includes: The point cloud acquisition module is used to acquire the spatial point cloud of the object to be cleaned; The plane determination module is used to determine the cutting plane of the point cloud region where the spatial point cloud is located; A point cloud projection module is used to project the spatial point cloud onto the cutting plane to obtain the projection points of the spatial point cloud on the cutting plane. The planar path module is used to determine the planar coverage path of the projection points on the cutting plane, cluster the projection points on the cutting plane to obtain clusters of the projection points; determine a first projection point, a second projection point, and a first direction vector among the projection points in the clusters according to the search grid on the cutting plane; the distance between the second projection point and the first projection point meets a fourth preset condition; the first direction vector is the direction vector between the first projection point and the second projection point; determine a third projection point and a second direction vector according to the search grid of the neighborhood of the second projection point; the second direction vector is the direction vector between the second projection point and the third projection point; the angle between the second direction vector and the first direction vector meets a fifth preset condition; determine the number of grids to be searched in the search grid, take the third projection point as the new second projection point, take the second direction vector as the new first direction vector, and return to the step of determining the third projection point and the second direction vector according to the search grid of the neighborhood of the second projection point, until the number of grids to be searched meets a sixth preset condition, and obtain the planar coverage path of the projection points on the cutting plane; The spatial path module is used to determine the spatial coverage path of the object to be cleaned based on the planar coverage path.

9. The apparatus according to claim 8, characterized in that, The point cloud acquisition module is also used to acquire the original depth point cloud information of the target area; the target area includes the object to be cleaned; and to perform three-dimensional reconstruction based on the original depth point cloud information to obtain the target spatial point cloud corresponding to the original depth point cloud information. The target spatial point cloud is segmented to obtain the spatial point cloud of the object to be cleaned.

10. The apparatus according to claim 8, characterized in that, The plane determination module is further configured to determine the origin of the cutting plane and the first coordinate axis associated with the origin; perform a cross product between the first coordinate axis and an auxiliary vector to obtain a second coordinate axis; the angle between the auxiliary vector and the first coordinate axis meets a first preset condition; The first coordinate axis and the second coordinate axis are cross-producted to obtain the third coordinate axis; the cutting plane is determined based on the origin, the first coordinate axis, the second coordinate axis and the third coordinate axis.

11. An electronic device, characterized in that, include: Memory, one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the overlay path generation method as described in any one of claims 1-7.

12. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the overlay path generation method as described in any one of claims 1-7.

Images