Positioning method, positioning device, computer device and computer storage medium of self-moving device

Abstract

The application relates to a positioning method and device of a self-moving device, a computer device and a computer readable storage medium. The method comprises the following steps: when the self-moving device enters a low space, acquiring first environment point cloud data collected by the self-moving device; determining a prior three-dimensional map based on a prior grid map determined by the self-moving device; and determining the pose of the self-moving device by using the first environment point cloud data and the prior three-dimensional map. The method can improve the positioning accuracy of the self-moving device in the low space.

Description

Technical Field

[0001] This application relates to the field of self-moving device technology, and in particular to a positioning method, positioning device, computer equipment, computer-readable storage medium, and computer program product for self-moving devices. Background Technology

[0002] When self-moving cleaning devices, such as robotic vacuum cleaners, enter low spaces like under sofas or beds, the limited gap between the top of the device and the top edge of the space, coupled with the fact that the LiDAR sensor is located on top of the device, prevents it from entering and detecting the space, thus hindering its localization. Existing solutions attempt to lower the LiDAR sensor to collect environmental point cloud data for localization; however, this lowering of the LiDAR sensor results in low localization accuracy. Summary of the Invention

[0003] Therefore, it is necessary to provide a positioning method, positioning device, computer device, computer-readable storage medium, and computer program product for self-moving devices that can improve positioning accuracy.

[0004] In a first aspect, this application provides a positioning method for a self-moving device, comprising:

[0005] When the self-moving device enters a low-ceilinged space, the first environmental point cloud data collected by the self-moving device is acquired.

[0006] Based on the prior grid map determined by the self-moving device, a prior 3D map is determined;

[0007] The pose of the self-moving device is determined using the first environmental point cloud data and the prior 3D map.

[0008] Secondly, this application also provides a positioning device, comprising:

[0009] The acquisition module is used to acquire the first environmental point cloud data collected by the self-moving device when the self-moving device enters a low-profile space;

[0010] The determination module is used to determine a prior 3D map based on the prior grid map determined by the self-moving device;

[0011] The target module is used to determine the pose of the self-moving device using the first environmental point cloud data and the prior 3D map.

[0012] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described in the first aspect.

[0013] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.

[0014] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0015] The aforementioned positioning method, positioning device, computer equipment, computer-readable storage medium, and computer program product for self-moving devices, when the self-moving device enters a low-lying space, collects first environmental point cloud data based on the self-moving device without the participation of a laser ranging module. At the same time, a priori 3D map is determined based on the prior grid map determined by the self-moving device to reduce computational overhead. Subsequently, the pose of the self-moving device is determined using the first environmental point cloud data and the priori 3D map, thereby significantly improving positioning accuracy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an application environment diagram of a self-moving device positioning method in one embodiment;

[0018] Figure 2 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0019] Figure 3 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0020] Figure 4 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0021] Figure 5 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0022] Figure 6 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0023] Figure 7 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0024] Figure 8This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0025] Figure 9 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0026] Figure 10 This is a flowchart illustrating a positioning method for a self-moving device in one embodiment;

[0027] Figure 11 This is a structural block diagram of the positioning device in one embodiment;

[0028] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] The self-mobile device positioning method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed on the cloud or other network servers. Both the terminal and the server can be used independently to execute the self-moving device positioning method provided in this embodiment. The terminal and server can also work together to execute the self-moving device positioning method provided in this embodiment. For example, terminal 102 executes the following: when the self-moving device enters a low-ceilinged space, it acquires first environmental point cloud data collected by the self-moving device; based on the prior grid map determined by the self-moving device, it determines a prior 3D map; and using the first environmental point cloud data and the prior 3D map, it determines the pose of the self-moving device. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0031] In one embodiment, such as Figure 2 As shown, a positioning method for a self-moving device is provided. This embodiment uses the application of this method to a computer device as an example for illustration, including steps 202 to 204.

[0032] Step 202: When the self-moving device enters a low-ceilinged space, acquire the first environmental point cloud data collected by the self-moving device;

[0033] In this context, "low-profile space" refers to an area that the self-moving device cannot enter when the laser ranging module is in the raised state, or where entering the area would pose a risk of scratching the laser ranging module, but can enter when the laser ranging module is in the lowered state. For example, space above a predetermined distance from the top of the self-moving device can be considered a low-profile space. Specifically, when the laser ranging module is in the raised state, it must at least partially protrude from the top surface of the self-moving device; when the laser ranging module is in the lowered state, it must not protrude from the top surface of the self-moving device.

[0034] When the self-moving device is located in a low-lying space, the laser ranging module on the self-moving device is lowered to allow it to enter the low-lying space and acquire first environmental point cloud data collected by the camera module on the self-moving device. This first environmental point cloud data is three-dimensional point cloud data. The camera module is fixedly mounted at the front of the self-moving device. Therefore, in a low-lying space, the self-moving device does not need to collect environmental point cloud data through the laser ranging module, thus eliminating the positioning inaccuracy caused by the raising and lowering of the laser ranging module and improving positioning accuracy. In addition, compared with the two-dimensional environmental point cloud data generated by the laser ranging module, the first environmental point cloud data collected by the camera module in this embodiment is three-dimensional point cloud data, which adds the height dimension. This provides complete geometric information for positioning in low-lying spaces, significantly reducing the negative impact of sparse environmental features or geometric degradation on positioning in low-lying spaces. This makes the positioning of the self-moving device in low-lying spaces more accurate and robust, thereby further improving its positioning accuracy.

[0035] For example, the height of space a is 130mm. When the laser ranging module is in the raised state, the overall height of the self-moving device is 131mm, which cannot enter space a. When the laser ranging module is in the lowered state, the overall height of the self-moving device is 128mm, which can enter space a. Therefore, space a is a low-profile space.

[0036] In one embodiment, when the self-moving device enters a low-ceilinged space, first environmental point cloud data is generated based on the depth image captured by the camera module. The depth image stores the pixel coordinates and depth value corresponding to each pixel, where the depth represents the distance between the object and the camera module. Combining the intrinsic parameters of the camera module, including focal length and optical center coordinates, the pixel coordinates of each pixel can be back-projected into three-dimensional space using a projection formula to obtain the 3D point cloud coordinates corresponding to the pixel. The normal vector of the pixel is obtained by performing a cross product calculation on the 3D point cloud coordinates corresponding to the pixel in the depth image and the 3D point cloud coordinates corresponding to its neighboring pixels, where neighboring pixels include the pixel's lower-right neighbor or upper-left neighbor. The 3D point cloud coordinates and normal vectors corresponding to all pixels are combined to form the first environmental point cloud data. It should be noted that in this embodiment, the coordinate system of the self-moving device is defined as follows: the X-axis points to the left or right of the self-moving device, the Y-axis points to the forward direction of the self-moving device, and the Z-axis points vertically upward, corresponding to the height direction of the self-moving device. Therefore, in the 3D point cloud coordinates (x, y, z) corresponding to each pixel, y is called the depth value and z is called the height value, which will not be elaborated further below.

[0037] In one embodiment, the camera module can be a binocular vision camera or a monocular vision camera; this embodiment is not limited in this regard, and those skilled in the art can use any camera capable of acquiring three-dimensional environmental point cloud data. The laser ranging module can be a Laser Distance Sensor (LDS) or other types of ranging sensors; this embodiment is not limited in this regard.

[0038] Step 204: Determine a priori 3D map based on the prior grid map determined by the self-moving device;

[0039] The prior grid map is a two-dimensional grid map, an environmental model representing the spatial structure of the work environment in grid form, which is acquired or loaded by the self-device before it starts running. It can be understood that the prior grid map divides the continuous spatial structure of the work environment into a regular grid matrix composed of a large number of equally sized grids. Each grid uses an occupancy probability to represent the likelihood that its location is occupied by an obstacle, and each grid has corresponding 2D coordinates. By upscaling the prior grid map to obtain a prior three-dimensional map, computational overhead can be reduced, eliminating the need to reconstruct a new three-dimensional map.

[0040] In one embodiment, after each successful positioning, the mobile device updates the currently collected environmental point cloud data to the prior grid map, thereby continuously improving the accuracy of the prior grid map. Therefore, the prior grid map in this embodiment can be the prior grid map updated after the most recent successful positioning.

[0041] Step 206: Determine the pose of the self-moving device using the first environmental point cloud data and the prior 3D map.

[0042] Specifically, the first environmental point cloud data is mapped to a prior 3D map for matching through candidate poses, and the pose of the self-moving device is determined from the candidate poses based on the matching results. Candidate poses include poses generated based on prior poses, and prior poses include poses obtained from the self-moving device's historical localization.

[0043] Since the actual pose of a self-moving device during continuous movement typically does not deviate too far from the prior pose, the candidate pose is generated based on the prior pose and located near it. By matching the pose of the first environmental point cloud data acquired by the camera module in low-ceilinged spaces with the prior 3D map, complete perception of low-ceilinged spaces can be achieved, thereby significantly improving the positioning accuracy of the self-moving device. Specifically, by matching the pose of the first environmental point cloud data with the prior 3D map, TOF (Time of Flight) / LDS (Laser Distance Sensor) level accuracy can be achieved in low-ceilinged spaces.

[0044] In one embodiment, after the first environmental point cloud data is mapped to the prior 3D map through the candidate pose, the target 3D point cloud obtained by mapping the first environmental point cloud data is matched with the target reference 3D point cloud in the prior 3D map to obtain the matching result of the candidate pose, and then the pose of the self-moving device is determined from multiple candidate poses based on the matching result.

[0045] The target reference 3D point cloud includes a reference 3D point cloud in the prior 3D map that is located within a preset distance range of the target 3D point cloud. By determining the matching result of the candidate poses based on the difference between the target 3D point cloud and the target reference 3D point cloud, the pose of the self-moving device is determined from multiple candidate poses based on the matching result.

[0046] In one embodiment, after step 206, the method further includes: updating the first environmental point cloud data to the prior 3D map, thereby continuously improving the quality of the prior 3D map and improving the positioning accuracy.

[0047] The aforementioned self-moving device positioning method, when the self-moving device enters a low-ceilinged space, collects first environmental point cloud data based on the self-moving device, without the need for a laser ranging module. At the same time, it determines a prior 3D map based on the prior grid map determined by the self-moving device to reduce computational overhead. Then, it matches the first environmental point cloud data and the prior 3D map to determine the pose of the self-moving device, thereby significantly improving positioning accuracy.

[0048] In an exemplary embodiment, the self-mobile device localization method further includes: when the self-mobile device leaves a low-ceilinged space, acquiring second environmental point cloud data collected by the self-mobile device, and using the second environmental point cloud data and a priori grid map to determine the pose of the self-mobile device.

[0049] In this process, when the self-moving device leaves the low-ceilinged space, the laser ranging module on the self-moving device is raised to acquire second environmental point cloud data collected by the laser ranging module. This second environmental point cloud data is two-dimensional point cloud data, and the prior grid map is also a two-dimensional grid map. The second environmental point cloud data is then directly mapped to the prior grid map through candidate poses for matching. Based on the matching results, the pose of the self-moving device is determined from the candidate poses. Similarly, the generation of candidate poses can refer to any of the above embodiments, and will not be repeated here. After the self-moving device leaves the low-ceilinged space, the space height is sufficient to accommodate the laser ranging module in a raised state, and the environmental structural features are rich. Therefore, using the second environmental point cloud data and the prior grid map to directly determine the pose of the self-moving device can reduce computational overhead and ensure the accuracy and real-time performance of positioning.

[0050] In this embodiment, when the self-moving device leaves the low-profile space, the second environmental point cloud data collected by the self-moving device is acquired. The pose of the self-moving device is determined by using the second environmental point cloud data and the prior grid map, so that the self-moving device can achieve accurate and real-time positioning of the self-moving device with low computational overhead in non-low-profile spaces.

[0051] In one exemplary embodiment, such as Figure 3 As shown, determining the prior 3D map based on the prior grid map determined by the self-moving device also includes steps 302 to 304.

[0052] Step 302: Upscale the 2D coordinates corresponding to the obstacle grids in the prior grid map to obtain 3D point cloud coordinates;

[0053] In this process, based on the occupancy probability of each grid cell in the prior grid map, it can be determined whether the grid cell is an obstacle grid cell. Then, the height value z of the 2D coordinates corresponding to the obstacle grid cell is interpolated to complete the height value z in the 2D coordinates, thereby obtaining the 3D point cloud coordinates and achieving dimensionality upgrade.

[0054] In one embodiment, for the 2D coordinates corresponding to each obstacle grid, 3D point cloud coordinates are obtained by interpolating at equal intervals with a preset step size within the detection height range of the laser ranging module. Performing equal-interval interpolation within the detection height range of the laser ranging module ensures that the height value z of the obtained 3D point cloud coordinates closely matches the actual height distribution of objects in the real scene, thereby further improving positioning accuracy.

[0055] In one embodiment, the detection height range of the laser ranging module is [3, 15cm], and the preset step size is 1cm. Therefore, for the 2D coordinates corresponding to each obstacle grid, the 3D point cloud coordinates are obtained by interpolating at equal intervals within [3, 15cm] with a step size of 1cm.

[0056] In one embodiment, before upscaling the 2D coordinates corresponding to obstacle grids in the prior grid map to obtain 3D point cloud coordinates, the method further includes: selecting a sampling origin in the passable area of ​​the prior grid map; wherein the passable area includes areas whose occupancy probability is less than a preset probability threshold, and grids through which the sampling ray passes and whose occupancy probability is greater than the preset probability threshold are determined as obstacle grids, wherein the sampling ray is a virtual ray with the sampling origin as its endpoint.

[0057] In the passable area, a sampling origin is selected at preset step lengths, including 1m.

[0058] For example, if the sampling ray passes through grid a, and its probability of being occupied is 0.8, which is less than a preset probability threshold, then grid a is determined to be an obstacle grid.

[0059] Step 304: Rotate the normal vector of the 3D point cloud coordinates to face the target direction to obtain the prior 3D map.

[0060] Specifically, the normal vectors of all 3D point cloud coordinates are rotated to face the target direction, ensuring that all normal vectors have a consistent orientation. This eliminates ambiguity caused by the bidirectional nature of normal vectors, thereby improving positioning reliability. The target direction includes the direction towards the sampling origin, but this embodiment does not limit this.

[0061] In one embodiment, after rotating the normal vector of the 3D point cloud coordinates to face the target direction to obtain the prior 3D map, the method further includes: performing voxelization on the prior 3D map to reduce the amount of map data and further reduce computational overhead.

[0062] For example, the prior 3D map can be divided into discrete voxel grids, with each voxel grid retaining only one 3D point cloud coordinate. The retained 3D point cloud coordinate includes the 3D point cloud coordinate corresponding to the geometric center within the voxel grid or the centroid of the 3D point cloud within the voxel grid.

[0063] In this embodiment, the 2D point cloud coordinates corresponding to obstacle grids in the prior grid map are upgraded to obtain 3D point cloud coordinates. The normal vectors of the 3D point cloud coordinates are rotated to face the target direction, thereby obtaining a prior 3D map. There is no need to rebuild the 3D map. Instead, the existing 2D prior grid map is directly used for data upgrade. By retaining only the 3D point cloud coordinates corresponding to obstacle grids for subsequent 3D matching, the computational overhead is significantly reduced and the positioning accuracy is improved.

[0064] In one exemplary embodiment, such as Figure 4 As shown, obtaining the first environmental point cloud data collected from the mobile device also includes steps 402 to 412.

[0065] Step 402: Binarize the depth image acquired by the self-moving device to generate a binarized image;

[0066] The depth image contains invalid depth pixels that are useless for pose matching. These include depth loss points or noise points caused by camera module blind spots, specular reflections, transparent objects, or distant backgrounds. After binarization, these invalid depth pixels are filtered out, and only valid depth pixels are retained. In other words, pixels with a value of 0 in the binarized image are invalid depth pixels, while pixels with a value of 1 (non-zero pixels) are valid pixels.

[0067] In one embodiment, after binarizing the depth image acquired by the mobile device to generate a binarized image, the method further includes: in the binarized image, removing pixels whose height value z exceeds a preset height value range and whose depth value y exceeds a preset depth value range based on the 3D point cloud coordinates corresponding to each pixel; the preset height value range and the preset depth value range include the detection range of the camera module.

[0068] The camera module has a limited detection range. Pixels outside this range have invalid depth values, lacking useful information, and are therefore removed to further reduce computational overhead and improve positioning accuracy. The preset height range is [2, 38] cm, and the preset depth range is [2, 120] cm. It should be noted that pixel removal is achieved by marking the corresponding pixel value in the binarized image as 0. The pixel removal described in the following steps follows the same principle and will not be repeated here.

[0069] Step 404: Cluster adjacent pixels in the depth image that have the same normal vector direction and continuous depth values ​​to obtain at least two connected components;

[0070] The depth image includes planar structures such as the ground, ceiling, and walls. In low-ceilinged spaces, the self-moving device cannot be located on the ground or ceiling, therefore its pose cannot lie within these planes. These planes constitute interference information for pose matching. By partitioning the depth image into connected components, the pixel data corresponding to the ground, ceiling, and walls can be distinguished, thus facilitating the subsequent removal of data from these planes.

[0071] In one embodiment, clustering adjacent pixels in the depth image with consistent normal vector directions and continuous depth values ​​to obtain at least two connected components further includes: for each seed pixel in the depth image, comparing the normal vector of each adjacent pixel in the neighborhood direction of the seed pixel with the normal vector of the seed pixel, wherein the adjacent pixels are pixels not marked as connected components; if the difference between the normal vector of the adjacent pixel and the normal vector of the seed pixel is greater than or equal to a first difference threshold, then the adjacent pixels are marked as different connected components, and the adjacent pixels are used as new seed pixels; if the difference between the normal vector of the adjacent pixel and the normal vector of the seed pixel is less than the first difference threshold, then the adjacent pixels and the seed pixel are marked as the same connected component.

[0072] The neighborhood directions of the seed pixel include the top-left, top-right, bottom-left, and bottom-right neighborhood directions. The seed pixel can be randomly selected from the depth image; this embodiment does not impose any limitations on this.

[0073] In one embodiment, by comparing the normal vectors of adjacent pixels with the normal vector of the seed pixel, if the angle between the normal vectors of adjacent pixels and the seed pixel is greater than or equal to a first angle threshold, it indicates that the normal vector directions are inconsistent, and the adjacent pixels are marked as different connected components. If the angle between the normal vectors of adjacent pixels and the seed pixel is less than the first angle threshold, it indicates that the normal vector directions are consistent, and the adjacent pixels and the seed pixel are marked as the same connected component.

[0074] The above method enables precise clustering of depth-continuous pixels in a depth image based on the consistency of the normal vector direction, thereby effectively separating planes or object surfaces with different orientations.

[0075] For example, the 3D point cloud coordinates of the seed pixel are (x1, y1, z1), and the 3D point cloud coordinates of its nth neighboring pixel in the upper left neighborhood direction are represented as follows: , This represents the spatial step size between adjacent pixels along the top-left neighborhood direction.

[0076] In one embodiment, to facilitate the differentiation of different connected domains, a unified preset identifier can be set for each pixel in a connected domain. This preset identifier is used to quickly index the 3D point cloud coordinates of all pixels in the corresponding connected domain. Subsequently, when processing the connected domain, the 3D point cloud coordinates of all pixels in the connected domain can be quickly obtained through the preset identifier, thereby improving computational efficiency.

[0077] In one embodiment, after clustering adjacent pixels in the depth image with consistent normal vector directions and continuous depth values ​​to obtain at least two connected components, the method further includes: removing connected components with fewer than a preset number of pixels. These connected components with too few pixels are false regions caused by noise, camera module errors, or random isolated points, which can introduce interference to pose matching. Removing them can reduce computational overhead and improve positioning accuracy.

[0078] Step 406: Perform principal component analysis on the connected component to obtain the feature vector of the connected component and the feature value corresponding to the feature vector; wherein the feature vector represents the main direction of pixel dispersion in the connected component, and the feature value represents the degree of dispersion of the connected component in the main direction corresponding to the feature vector.

[0079] Step 408: Determine the plane normal vector from the eigenvectors based on the eigenvalues;

[0080] In this process, a 3x3 covariance matrix is ​​calculated based on the 3D point cloud coordinates of all pixels within the connected component. Principal Component Analysis (PCA) is then performed on the covariance matrix to obtain the eigenvectors and their corresponding eigenvalues ​​for the connected component. Smaller eigenvalues ​​indicate less dispersion, while larger eigenvalues ​​indicate greater dispersion. Therefore, the principal direction represented by the eigenvector corresponding to the smallest eigenvalue is the direction in which the point cloud distribution is most concentrated. For a point cloud that approximates a plane, this direction is the plane normal vector direction. Therefore, the eigenvector corresponding to the smallest eigenvalue is taken as the plane normal vector of the connected component.

[0081] Step 410: If the plane normal vector of the connected component is vertical, then remove the corresponding pixel in the connected component from the binarized image.

[0082] In this approach, if the plane normal vector of a connected component is vertical, it indicates that the plane containing the connected component is the ground or ceiling plane. This enables accurate identification of various planes, allowing all pixels within the connected component to be removed from the binarized image, thus retaining only pixels representing wall surfaces. The vertical direction can be understood as the z-axis direction, corresponding to the height direction of the mobile device.

[0083] In one embodiment, if the absolute value of the height value in the plane normal vector of a connected component is greater than a preset height threshold, then the plane normal vector is determined to be vertical. It can be understood that the closer the absolute value of the height value is to 1, the closer the direction of the plane normal vector is to the vertical direction.

[0084] For example, the plane normal vector (0, 0, 0.8) of connected domain a has a height value of 0.8, which is greater than a preset height threshold, so the plane normal vector is determined to be in the vertical direction.

[0085] Step 412: Extract the 3D point cloud coordinates and normal vectors corresponding to each non-zero pixel in the binarized image to obtain the first environmental point cloud data.

[0086] In this process, after removing pixels from the ground and ceiling planes, the remaining non-zero pixels in the binarized image correspond to the pixels on the wall. Then, the 3D point cloud coordinates and normal vectors of the non-zero pixels are extracted for pose matching, thereby significantly reducing computational overhead and improving positioning accuracy.

[0087] In one embodiment, after extracting the 3D point cloud coordinates and normal vectors corresponding to non-zero pixels in the binarized image to obtain the first environment point cloud data, the method further includes: performing voxelization processing on the first environment point cloud data to reduce the amount of point cloud coordinate data and further reduce computational overhead.

[0088] For example, the first environmental point cloud data can be divided into discrete voxel grids, with each voxel grid retaining only one 3D point cloud coordinate. The retained 3D point cloud coordinate includes the 3D point cloud coordinate corresponding to the geometric center within the voxel grid or the centroid of the 3D point cloud within the voxel grid.

[0089] In this embodiment, the depth image acquired by the mobile device is binarized to generate a binarized image. Adjacent pixels with consistent normal vector directions and continuous depth values ​​in the depth image are clustered to obtain at least two connected components. Principal component analysis is then performed on the connected components to obtain the feature vectors and corresponding feature values ​​of the connected components. The planar normal vector of the connected component is then determined from the feature vectors based on the feature values. If the planar normal vector of the connected component is vertical, it indicates that the connected component is the ground and ceiling planes. In this case, the pixels corresponding to the connected component in the binarized image are removed. At this time, only the pixels of the vertical wall are retained in the binarized image. Then, the 3D point cloud coordinates and normal vectors of each non-zero pixel are extracted from the binarized image to obtain the first environmental point cloud data containing only the wall surfaces. This achieves accurate removal of invalid 3D point clouds corresponding to the ground and ceiling planes, thereby significantly reducing computational overhead and improving positioning accuracy.

[0090] In one exemplary embodiment, such as Figure 5As shown, after determining the plane normal vector of the connected region from the eigenvector based on the eigenvalue, the method further includes steps 502 to 508.

[0091] Step 502: Construct a fitting plane based on the 3D point cloud within the connected domain and the plane normal vector;

[0092] Among them, thin strip-shaped areas with excessively small physical dimensions, such as door gaps, cables, and small fragment planes, do not represent planar structures with independent semantic meaning in the physical world and lack practical application value. Retaining such planes would interfere with pose matching. Therefore, such planes are eliminated to further reduce computational overhead and improve positioning accuracy.

[0093] In one embodiment, the centroid of the connected domain can be calculated based on the 3D point cloud within the connected domain, and a fitting plane for the connected domain can be constructed based on the centroid and the plane normal vector. The operation of the fitting plane is defined as follows:

[0094]

[0095] in, Represents the plane normal vector. This represents the 3D point cloud coordinates of any point on the fitted plane. Indicates the center of mass.

[0096] Step 504: Project the pixels within the connected region onto the fitting plane;

[0097] In one embodiment, before projecting the pixels within the connected domain onto the fitting plane, the method further includes: calculating the distance from the 3D point cloud coordinates of each pixel within the connected domain to the fitting plane; if the distance is greater than a preset distance threshold, then the corresponding pixel in the binarized image is removed.

[0098] Uneven pixels, such as those on curved surfaces and scattered points, cannot provide consistent and repeatable reference information for matching and localization, thus interfering with pose matching. Therefore, such planes and point groups are eliminated to further reduce computational overhead and improve localization accuracy. Uneven pixels refer to pixels whose observed position and shape change significantly over time or with changes in the camera module's viewing angle. By measuring the distance from the 3D point cloud coordinates of the pixel to the fitting plane, these uneven pixels can be accurately identified and subsequently removed from the binarized image.

[0099] In one embodiment, the root mean square of the 3D point cloud within the connected region to the fitted plane can be calculated; if the root mean square is greater than a preset root mean square threshold, the pixels corresponding to the connected region in the binarized image are removed.

[0100] The root mean square (RMS) reflects the overall flatness of the connected region. If the RMS is greater than a preset RMS threshold, it indicates that the pixels within the connected region are spatially scattered or curved. This method can further improve the accuracy of identifying uneven pixels, thereby effectively reducing the computational overhead of subsequent algorithms and improving overall localization accuracy. The preset RMS threshold includes 3 cm.

[0101] Step 506: Calculate the convex hull area of ​​the connected region and the minimum bounding rectangle corresponding to the convex hull in the fitting plane;

[0102] Step 508: If the convex hull area of ​​the connected component and the minimum bounding rectangle satisfy a preset condition, then the pixels corresponding to the connected component in the binarized image are removed.

[0103] The preset conditions include a convex hull area smaller than a preset area threshold and / or the shorter side of the smallest bounding rectangle being smaller than a preset short side threshold. If these preset conditions are met, it indicates that the connected component is a thin, strip-shaped region with an excessively small physical size. Pixels corresponding to this connected component in the binary image are then removed, further reducing computational overhead and improving localization accuracy. The preset area threshold includes 0.02m. 2 The preset short side threshold includes 0.06m.

[0104] In this embodiment, a fitting plane is constructed based on the 3D point cloud and plane normal vectors within the connected region. The pixels within the connected region are projected onto the fitting plane. Then, the convex hull area and the minimum bounding rectangle corresponding to the convex hull of the connected region are calculated on the fitting plane. If the convex hull area and the minimum bounding rectangle of the connected region meet the preset conditions, it indicates that the connected region is a thin strip-shaped region with an excessively small physical size. The pixels corresponding to the connected region in the binarized image are then removed, achieving accurate removal of invalid 3D point clouds. This further significantly reduces computational overhead and improves positioning accuracy.

[0105] In one exemplary embodiment, such as Figure 6 As shown, the positioning method for the self-moving device further includes steps 602 to 604.

[0106] Step 602: Extract the contours of each non-zero region in the binarized image to obtain the outer contour of each non-zero region, and calculate the convex hull area and pixel area of ​​the outer contour.

[0107] In this study, non-convex point sets in the binarized image cannot represent continuous physical planes in the real world. If used for pose matching, they will cause pose drift. Therefore, they need to be removed to further reduce computational overhead and improve positioning accuracy. A non-convex point set refers to a set of point clouds with depressions, gaps, or internal holes. Contour extraction is performed on each non-zero region in the binarized image to obtain the outer contour of each non-zero region. For non-convex point sets, the outer contour is usually L-shaped, C-shaped, or a ring with a hole in the middle. Then, the convex hull area and the pixel area of ​​the outer contour are calculated. Based on the difference between the convex hull area and the pixel area, it can be determined whether the point set in the non-zero region is a non-convex point set. The above step 602 can be executed after step 402 or after step 508. This embodiment does not specifically limit this.

[0108] Step 604: If the difference between the convex hull area and the pixel area is greater than the second difference threshold, then the pixels corresponding to the non-zero region in the binarized image are removed.

[0109] The difference includes the difference or ratio between the convex hull area and the pixel area, and the corresponding second difference threshold may include a difference threshold or a ratio threshold. If the difference between the convex hull area and the pixel area of ​​a non-zero region is greater than the second difference threshold, it indicates that the point set of the non-zero region is a non-convex point set, and the pixels in the non-zero region in the binarized image are then removed, thereby further reducing computational overhead and improving positioning accuracy.

[0110] In this embodiment, the contours of each non-zero region in the binarized image are extracted to obtain the outer contour of each non-zero region. The convex hull area and pixel area of ​​the outer contour are calculated. If the difference between the convex hull area and the pixel area is greater than a preset difference threshold, the pixels of the corresponding non-zero region in the binarized image are removed, thereby achieving accurate removal of invalid 3D point clouds, which further significantly reduces the computational overhead and improves the positioning accuracy.

[0111] In one exemplary embodiment, such as Figure 7 As shown, determining the pose of the self-moving device using the first environmental point cloud data and the prior 3D map also includes steps 702 to 706.

[0112] Step 702: Globally enumerate the prior pose to generate a first candidate pose;

[0113] Step 704: Perform local iterative optimization on the prior pose to generate a second candidate pose;

[0114] In this context, the true pose of a self-moving device during continuous motion typically doesn't deviate too far from the prior pose. However, there's no prior knowledge about the distance between the self-moving device's pose and the prior pose; therefore, the selection of candidate poses is crucial. If candidate poses are inappropriately selected and fail to cover the area where the true pose is located, the matched pose will deviate from the true value, leading to localization failure. Here, global enumeration of the prior pose generates a first candidate pose that is far from the prior pose, while local iterative optimization of the prior pose generates a second candidate pose that is closer to the prior pose. This ensures that candidate poses cover the range of the true pose, thus avoiding missing the true pose and improving localization accuracy.

[0115] In one embodiment, the prior pose is globally enumerated according to a preset step size to generate a first candidate pose, and the prior pose is locally optimized by iteratively solving the pose increment of the prior pose to generate a second candidate pose.

[0116] Step 706: Map the first environmental point cloud data to the prior 3D map through the first candidate pose and the second candidate pose respectively for matching, and determine the pose of the self-moving device from the first candidate pose and the second candidate pose based on the matching result.

[0117] The first environmental point cloud data is in the coordinate system of the self-moving device, which is inconsistent with the coordinate system of the prior 3D map. Therefore, the first environmental point cloud data needs to be mapped to the prior 3D map using candidate poses. This process transforms the first environmental point cloud data from the self-moving device coordinate system to the prior 3D map coordinate system. The mapped target 3D point cloud is then matched with the target reference 3D point cloud in the prior 3D map. Based on the matching result, the pose of the self-moving device is determined from the first and second candidate poses. Since the first and second candidate poses include poses that are far from and close to the prior pose, they can cover different deviations from the true pose, avoiding matching failures due to improper candidate pose selection, thereby significantly improving positioning accuracy.

[0118] In this embodiment, a first candidate pose is generated by globally enumerating the prior pose, and a second candidate pose is generated by performing local iterative optimization on the prior pose. This ensures that the candidate poses cover the range of the real pose. Then, the first environmental point cloud data is mapped to the prior 3D map for matching through the first and second candidate poses. Based on the matching results, the pose of the self-moving device is determined from the first and second candidate poses. This ensures that the target pose that is closest to the real pose can be determined from the candidate poses, further significantly improving the positioning accuracy.

[0119] In one exemplary embodiment, such as Figure 8As shown, the process of generating the first candidate pose by globally enumerating the prior pose further includes steps 802 to 806.

[0120] Step 802: Enumerate the prior pose according to the first preset step size to generate a third candidate pose; the third candidate pose includes the prior pose.

[0121] Specifically, the prior pose is coarsely enumerated according to the first preset step size to generate the third candidate pose, thereby performing a large-scale search around the prior pose to ensure coverage of all possible areas of the real pose.

[0122] Step 804: The first environmental point cloud data is mapped to the prior 3D map through the third candidate pose for matching, and the fourth candidate pose is determined from the third candidate pose based on the matching result;

[0123] Specifically, by determining the difference between the fourth 3D point cloud obtained after mapping the first environmental point cloud data and the fourth reference 3D point cloud, the matching result of the candidate pose is determined. Then, the candidate pose with the best matching result is selected from multiple third candidate poses as the fourth candidate pose, thereby obtaining a coarse matching result that is close to the real pose. The fourth reference 3D point cloud includes reference 3D point clouds in the prior 3D map that are located within a preset distance range of the fourth 3D point cloud.

[0124] Step 806: Enumerate the fourth candidate pose according to the second preset step size to generate the first candidate pose; wherein the second preset step size is smaller than the first preset step size.

[0125] The second preset step size is smaller than the first preset step size, which allows for fine enumeration near the coarse matching result, i.e., the fourth candidate pose, thus achieving coarse-to-fine pose enumeration and avoiding the omission of candidate poses that are closer to the real pose due to the first preset step size being too large, thereby further improving the positioning efficiency and positioning accuracy.

[0126] In this embodiment, a third candidate pose is generated by enumerating the prior pose according to a first preset step size. The first environmental point cloud data is then mapped to the prior 3D map for matching using the third candidate pose. Based on the matching result, a fourth candidate pose is determined from the third candidate pose. The fourth candidate pose is then enumerated according to a second preset step size to generate a first candidate pose. The second preset step size is smaller than the first preset step size, thereby achieving coarse-to-fine pose enumeration and avoiding the omission of candidate poses that are close to the true pose, thus improving positioning efficiency and positioning accuracy.

[0127] In one exemplary embodiment, such as Figure 9 As shown, the process of performing local iterative optimization on the prior pose to generate the second candidate pose also includes steps 902 to 908.

[0128] Step 902: Use the prior pose as the initial pose;

[0129] Step 904: Map the first environmental point cloud data to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud; determine the pose increment of the initial pose based on the difference between the first 3D point cloud and the first reference 3D point cloud in the prior 3D map, wherein the first reference 3D point cloud is a reference 3D point cloud in the prior 3D map that is located within a preset distance range of the first 3D point cloud.

[0130] Specifically, the difference between the first 3D point cloud and the first reference 3D point cloud is used as the error vector. The partial derivative of the error vector with respect to the preset pose parameters is calculated to obtain the Jacobian matrix. Then, the incremental equation composed of the Jacobian matrix and the error vector is solved to obtain the pose increment.

[0131] Step 906: Fuse the initial pose with the pose increment to obtain the updated pose;

[0132] Step 908: Using the updated pose as the initial pose, continue to execute the step of mapping the 3D point cloud to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud, until the updated pose converges; use the converged pose as the second candidate pose.

[0133] In this process, by continuously updating the pose locally, the pose gradually approaches the true pose, and finally obtains a second candidate pose that is closer to the prior pose and has higher accuracy. This can cover different deviations from the true pose, avoid matching failure due to improper selection of candidate poses, and thus significantly improve the positioning accuracy.

[0134] In this embodiment, by using the prior pose as the initial pose, the first environmental point cloud data is mapped to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud. Based on the difference between the first 3D point cloud and the first reference 3D point cloud in the prior 3D map, the pose increment of the initial pose is determined. The first reference 3D point cloud is a reference 3D point cloud in the prior 3D map located within a preset distance range of the first 3D point cloud. The initial pose and the pose increment are fused to obtain the updated pose. Then, the updated pose is used as the initial pose, and the step of mapping the first environmental point cloud data to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud is continued until the updated pose converges. The converged pose is used as the second candidate pose. Through the above-mentioned local iterative optimization processing method, a second candidate pose that is closer to the prior pose and has higher accuracy can be obtained, thereby covering different deviation ranges of the real pose and avoiding matching failure due to improper selection of candidate poses, thus further significantly improving the positioning accuracy.

[0135] In one exemplary embodiment, such as Figure 10 As shown, step 704 also includes steps 1002 to 1006.

[0136] Step 1002: Map the 3D point cloud to the prior 3D map through the first candidate pose to obtain the corresponding second 3D point cloud; map the 3D point cloud to the prior 3D map through the second candidate pose to obtain the corresponding third 3D point cloud.

[0137] Step 1004: Based on the difference between the second 3D point cloud and the second reference 3D point cloud in the prior 3D map, determine the matching degree of the corresponding first candidate pose; based on the difference between the third 3D point cloud and the third reference 3D point cloud in the prior 3D map, determine the matching degree of the second candidate pose, wherein the second reference 3D point cloud is a reference 3D point cloud in the prior 3D map located within a preset distance range of the second 3D point cloud, and the third reference 3D point cloud is a reference 3D point cloud in the prior 3D map located within a preset distance range of the third 3D point cloud;

[0138] The closer the candidate pose is to the current true pose of the mobile device, the smaller the difference between the 3D point cloud mapped from the candidate pose and the reference 3D point cloud; conversely, the greater the difference, the closer the candidate pose is to the reference 3D point cloud. Therefore, the matching degree of the first candidate pose can be determined by the difference between the second 3D point cloud and the second reference 3D point cloud. Similarly, the matching degree of the second candidate pose can be determined by the difference between the third 3D point cloud and the third reference 3D point cloud. The greater the difference, the lower the matching degree; the smaller the difference in depth values, the higher the matching degree.

[0139] Step 1006: Determine the current target pose of the self-moving device from the first candidate pose and the second candidate pose based on the matching degree.

[0140] Among them, the candidate pose with the highest matching degree is selected from the first candidate pose and the second candidate pose as the pose of the self-moving device, thereby improving the positioning accuracy.

[0141] In one embodiment, a target candidate pose whose matching degree meets a preset matching degree threshold is determined from a first candidate pose and a second candidate pose. Then, the candidate pose with the highest matching degree is selected from the target candidate pose as the pose of the self-moving device. The preset matching degree threshold is a pre-set minimum acceptable value for matching degree. If neither the first candidate pose nor the second candidate pose meets the preset matching degree threshold, the localization is determined to have failed.

[0142] In one embodiment, the reliability of the reference 3D point cloud in the prior 3D map also affects the localization. If the reference 3D point cloud itself has measurement errors or inaccurate data, it may cause reference 3D point clouds with large differences to be incorrectly considered to have small differences, thus leading to incorrect matching degree calculation. Therefore, the localization method for the self-moving device further includes: determining the reliability corresponding to the matching degree of the first candidate pose based on the depth value of the second reference 3D point cloud, and determining the reliability corresponding to the matching degree of the second candidate pose based on the depth value of the third reference 3D point cloud, wherein a larger depth value indicates higher reliability, and vice versa, a smaller depth value indicates lower reliability, and then determining the pose of the self-moving device from the first candidate pose and the second candidate pose based on the matching degree and the reliability.

[0143] Among them, the candidate pose with the highest overall matching degree and reliability can be selected as the pose of the self-moving device, thereby ensuring the accuracy and reliability of positioning.

[0144] In one embodiment, a target candidate pose whose matching degree satisfies a preset matching degree threshold is determined from the first candidate pose and the second candidate pose, and then the candidate pose with the highest reliability is selected from the target candidate pose as the pose of the self-moving device.

[0145] In one embodiment, the sum of the reciprocals of the depth values ​​y of all 3D point cloud coordinates in the reference 3D point cloud can be used as the reliability. The larger the sum of the reciprocals, the lower the reliability, and the smaller the sum of the reciprocals, the higher the reliability.

[0146] In one embodiment, a target candidate pose whose matching degree meets a preset matching degree threshold is determined from the first candidate pose and the second candidate pose. Then, a candidate pose whose reliability meets a preset reliability threshold is determined from the target candidate pose as the pose of the self-moving device, thereby ensuring that the selected pose has both high matching accuracy and is based on a high-quality reference 3D point cloud, thereby further significantly improving the accuracy and reliability of positioning.

[0147] In this embodiment, a second 3D point cloud is obtained by mapping the first environmental point cloud data to a priori 3D map through a first candidate pose. A third 3D point cloud is obtained by mapping the first environmental point cloud data to the priori 3D map through the second candidate pose. The matching degree of the first candidate pose is determined based on the difference between the second 3D point cloud and the second reference 3D point cloud in the priori 3D map. The matching degree of the second candidate pose is determined based on the difference between the third 3D point cloud and the third reference 3D point cloud in the priori 3D map. The second reference 3D point cloud is a reference 3D point cloud in the priori 3D map that is located within a preset distance range of the second 3D point cloud. The third reference 3D point cloud is a reference 3D point cloud in the priori 3D map that is located within a preset distance range of the third 3D point cloud. The pose of the self-moving device is then determined from the first and second candidate poses based on the matching degree, thereby improving the positioning accuracy.

[0148] In an exemplary embodiment, the localization method for the self-moving device further includes: if the low-ceiling space is a corridor space, then the candidate pose includes the pose in the degradation direction; if the low-ceiling space is not a corridor space, then the candidate pose includes the pose in all directions.

[0149] In this context, a corridor space refers to a long, narrow space with walls on both sides and an open area in front, such as a long, straight corridor. The degradation direction refers to the direction along the corridor. In a corridor space, the non-degradation direction is the wall, which is inaccessible to the mobile device. Candidate poses in this direction will interfere with subsequent pose matching. Therefore, if the low-ceilinged space is a corridor space, only pose matching in the degradation direction is performed, thereby further improving positioning accuracy.

[0150] In an exemplary embodiment, when the pose of the self-moving device cannot be determined using the first environmental point cloud data and the prior 3D map, i.e., the positioning fails, the low-lying space is determined to be an unknown space or a space entered for the first time. Then, the historical environmental point cloud data collected by the self-moving device is obtained; the pose of the self-moving device is determined using the first environmental point cloud data and the historical environmental point cloud data.

[0151] The Iterative Closest Point (ICP) algorithm is used to match the first environmental point cloud data with the historical environmental point cloud data to calculate the relative pose transformation. The relative pose transformation and the prior pose are then superimposed to obtain the pose of the self-moving device.

[0152] In one embodiment, after determining the pose of the self-moving device based on the relative pose transformation and the prior pose, the method further includes: storing the currently collected first environmental point cloud data as historical environmental point cloud data; if the self-moving device re-enters the low-ceiling space, continuing to use the first environmental point cloud data and historical environmental point cloud data to determine the pose of the self-moving device, thereby gradually improving the positioning accuracy by continuously accumulating environmental point cloud data in the low-ceiling space.

[0153] In one exemplary embodiment, the self-moving device includes a cleaning robot, such as a cleaning vacuum cleaner.

[0154] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0155] Based on the same inventive concept, this application also provides a positioning device for implementing the positioning method of the self-moving device described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more positioning device embodiments provided below can be found in the limitations of the positioning method of the self-moving device described above, and will not be repeated here.

[0156] In one embodiment, such as Figure 11 As shown, a positioning device is provided, including an acquisition module 1102, a determination module 1104, and a target module 1106, wherein:

[0157] The acquisition module 1102 is used to acquire the first environmental point cloud data collected by the self-moving device when the self-moving device enters a low-profile space;

[0158] The determination module 1104 is used to determine a prior 3D map based on the prior grid map determined by the self-moving device;

[0159] The target module 1106 is used to determine the pose of the self-moving device using the first environmental point cloud data and the prior 3D map.

[0160] Each module in the aforementioned positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0161] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a self-moving device positioning method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0162] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0166] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0167] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0168] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A positioning method for a self-moving device, characterized in that, The method includes: When the self-moving device enters a low-ceilinged space, the first environmental point cloud data collected by the self-moving device is acquired; Based on the prior grid map determined by the self-moving device, a prior 3D map is determined; The pose of the self-moving device is determined using the first environmental point cloud data and the prior 3D map.

2. The method according to claim 1, characterized in that, The method further includes: acquiring the first environmental point cloud data collected by the self-mobile device further includes: acquiring the first environmental point cloud data collected by the camera module on the self-mobile device.

3. The method according to claim 1, characterized in that, The determination of the prior 3D map based on the prior grid map determined by the self-moving device further includes: The 2D coordinates corresponding to the obstacle grids in the prior grid map are upgraded to obtain 3D point cloud coordinates. The normal vector of the 3D point cloud coordinates is rotated to face the target direction to obtain the prior 3D map.

4. The method according to claim 3, characterized in that, Before upscaling the 2D coordinates corresponding to obstacle grids in the prior grid map to obtain 3D point cloud coordinates, the process also includes: A sampling origin is selected in the passable area of ​​the prior grid map; wherein the passable area includes areas whose probability of being occupied is less than a preset probability threshold. The grid cells through which the sampling ray passes and whose probability of being occupied is greater than a preset probability threshold are identified as the obstacle grid cells; the sampling ray is a virtual ray with the sampling origin as its endpoint; wherein the target direction includes the direction toward the sampling origin.

5. The method according to claim 1, characterized in that, The step of acquiring the first environmental point cloud data collected by the self-mobile device also includes: The depth image acquired by the self-moving device is binarized to generate a binarized image; Cluster adjacent pixels in the depth image that have the same normal vector direction and continuous depth value to obtain at least two connected components; Principal component analysis is performed on the connected component to obtain the feature vector of the connected component and the feature value corresponding to the feature vector. The feature vector represents the main direction of pixel dispersion in the connected component, and the feature value represents the degree of dispersion of the connected component in the main direction corresponding to the feature vector. The plane normal vector of the connected region is determined from the eigenvector based on the eigenvalue; If the plane normal vector of the connected component is vertical, then the corresponding pixel in the connected component in the binarized image is removed. The 3D point cloud coordinates and normal vectors corresponding to each non-zero pixel are extracted from the binarized image to obtain the first environmental point cloud data.

6. The method according to claim 5, characterized in that, The step of clustering adjacent pixels in the depth image that have the same normal vector direction and continuous depth values ​​to obtain at least two connected components further includes: For each seed pixel in the depth image, the normal vector of each neighboring pixel in the neighborhood direction of the seed pixel is compared with the normal vector of the seed pixel, wherein the neighboring pixels are pixels that are not marked as connected components; If the difference between the normal vector of the adjacent pixel and the normal vector of the seed pixel is greater than or equal to a first difference threshold, then the adjacent pixel is marked as a different connected region, and the adjacent pixel is used as a new seed pixel. If the difference between the normal vector of the adjacent pixel and the normal vector of the seed pixel is less than a first difference threshold, then the adjacent pixel and the seed pixel are marked as the same connected component.

7. The method according to claim 5, characterized in that, After determining the plane normal vector of the connected component from the eigenvector based on the eigenvalue, the method further includes: A fitting plane is constructed based on the 3D point cloud within the connected domain and the plane normal vector; Project the 3D point cloud within the connected domain onto the fitting plane; Calculate the convex hull area of ​​the connected region and the minimum bounding rectangle corresponding to the convex hull in the fitting plane; If the convex hull area of ​​the connected component and the minimum bounding rectangle satisfy a preset condition, then the pixels corresponding to the connected component in the binarized image are removed.

8. The method according to claim 5, characterized in that, The method further includes: Contour extraction is performed on each non-zero region in the binarized image to obtain the outer contour of each non-zero region, and the convex hull area and pixel area of ​​the outer contour are calculated. If the difference between the convex hull area and the pixel area is greater than the second difference threshold, then the pixels corresponding to the non-zero region in the binarized image are removed.

9. The method according to claim 5, characterized in that, The process of binarizing the depth image acquired by the self-mobile device to generate a binarized image further includes: In the binarized image, based on the 3D point cloud coordinates corresponding to each pixel, pixels whose height values ​​exceed a preset height value range and whose depth values ​​exceed a preset depth value range are removed; wherein the preset height value range and the preset depth value range include the detection range of the camera module.

10. The method according to claim 1, characterized in that, The step of determining the pose of the self-moving device using the first environmental point cloud data and the prior 3D map further includes: Globally enumerate the prior poses to generate the first candidate pose; The prior pose is subjected to local iterative optimization to generate a second candidate pose; The first environmental point cloud data is mapped to the prior 3D map through the first candidate pose and the second candidate pose for matching, and the pose of the self-moving device is determined from the first candidate pose and the second candidate pose based on the matching result.

11. The method according to claim 10, characterized in that, The step of globally enumerating the prior pose to generate the first candidate pose also includes: The prior pose is enumerated according to the first preset step size to generate a third candidate pose; the third candidate pose includes the prior pose. The first environmental point cloud data is mapped to the prior 3D map through the third candidate pose for matching, and the fourth candidate pose is determined from the third candidate pose based on the matching result; The fourth candidate pose is enumerated according to the second preset step size to generate the first candidate pose; wherein the second preset step size is smaller than the first preset step size.

12. The method according to claim 10, characterized in that, The step of performing local iterative optimization on the prior pose to generate the second candidate pose further includes: Use the prior pose as the initial pose; The first environmental point cloud data is mapped to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud; The pose increment of the initial pose is determined based on the difference between the first 3D point cloud and the first reference 3D point cloud in the prior 3D map; wherein the first reference 3D point cloud includes the reference 3D point cloud in the prior 3D map that is located within a preset distance range of the first 3D point cloud. The initial pose and the pose increment are fused to obtain the updated pose; The updated pose is used as the initial pose, and the step of mapping the first environmental point cloud data to the prior 3D map through the initial pose to obtain the corresponding first 3D point cloud is continued until the updated pose converges; the converged pose is used as the second candidate pose.

13. The method according to claim 10, characterized in that, The step of mapping the first environmental point cloud data to the prior 3D map through the first candidate pose and the second candidate pose respectively for matching, and determining the pose of the self-moving device from the candidate poses based on the matching result, further includes: The first environmental point cloud data is mapped to the prior 3D map through the first candidate pose to obtain the corresponding second 3D point cloud; the first environmental point cloud data is mapped to the prior 3D map through the second candidate pose to obtain the corresponding third 3D point cloud. The matching degree of the corresponding first candidate pose is determined based on the difference between the second 3D point cloud and the second reference 3D point cloud in the prior 3D map; wherein the second reference 3D point cloud is a reference 3D point cloud in the prior 3D map that is located within a preset distance range of the second 3D point cloud. The matching degree of the second candidate pose is determined based on the difference between the third 3D point cloud and the third reference 3D point cloud in the prior 3D map; wherein the third reference 3D point cloud is a reference 3D point cloud in the prior 3D map that is located within a preset distance range of the third 3D point cloud. The pose of the self-moving device is determined from the first candidate pose and the second candidate pose based on the matching degree.

14. The method according to claim 13, characterized in that, The method further includes: Based on the depth value of the second reference 3D point cloud, the reliability corresponding to the matching degree of the first candidate pose is determined; based on the depth value of the third reference 3D point cloud, the reliability corresponding to the matching degree of the second candidate pose is determined. The pose of the self-moving device is determined from the first candidate pose and the second candidate pose based on the matching degree and the reliability.

15. The method according to claim 1, characterized in that, The method further includes: When the self-moving device leaves the low-profile space, acquire the second environmental point cloud data collected by the self-moving device; The pose of the self-moving device is determined using the second environmental point cloud data and the prior grid map.

16. The method according to claim 15, characterized in that, The step of acquiring the second environmental point cloud data collected by the self-mobile device further includes: acquiring the second environmental point cloud data collected by the laser ranging module on the self-mobile device.

17. The method according to claim 1, characterized in that, The method further includes: When the self-moving device is located in the low-profile space, the laser ranging module on the self-moving device is controlled to be in a descending state; wherein, in the descending state, the laser ranging module does not protrude from the top surface of the self-moving device; When the self-moving device leaves the low-profile space, the laser ranging module on the self-moving device is controlled to be in a raised state; wherein, in the raised state, the laser ranging module at least partially protrudes from the top surface of the self-moving device.

18. The method according to claim 1, characterized in that, The self-moving device includes a cleaning robot.

19. A positioning device, characterized in that, include: The acquisition module is used to acquire the first environmental point cloud data collected by the self-moving device when the self-moving device enters a low-profile space; The determination module is used to determine a prior 3D map based on the prior grid map determined by the self-moving device; The target module is used to determine the pose of the self-moving device using the first environmental point cloud data and the prior 3D map.

20. A computer device comprising a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 18.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 18.

Images