Positioning method and cleaning equipment

Abstract

The invention provides a positioning method and cleaning equipment, and the method comprises the steps: obtaining an environment image collected by an image collection device and point cloud data collected by a laser radar in response to a positioning demand of a cleaning task, and determining an initial pose corresponding to a current environment based on the environment image, and the initial pose is adjusted based on the point cloud data to obtain the current pose of the cleaning equipment, and the cleaning equipment is controlled to execute the cleaning task based on the current pose, so that the calculation amount, energy consumption and time cost required when the cleaning equipment is positioned can be reduced.

Description

Technical Field

[0001] This application relates to the field of cleaning equipment technology, and in particular to a positioning method and a cleaning device. Background Technology

[0002] Robotic vacuum cleaners and other cleaning devices are widely used in home environments. Their core function is to replace manual cleaning by autonomously navigating and adjusting cleaning strategies. However, in some cases, cleaning devices need to redetermine their position on a map through positioning.

[0003] In existing technologies, cleaning equipment is mainly located using point cloud data from LiDAR. However, this requires controlling the cleaning equipment to rotate a full circle to collect point cloud data, and also requires controlling the cleaning equipment to match different positions, which increases the computational load, energy consumption, and time cost required for positioning.

[0004] Therefore, how to enable cleaning equipment to be positioned more effectively is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] This application provides a positioning method and a cleaning device that enables the cleaning device to perform positioning more effectively, thereby reducing the computational load, energy consumption, and time costs required for positioning.

[0006] A first aspect of this application provides a positioning method applied to a cleaning device, the cleaning device including an image acquisition device and a lidar, the method comprising: in response to the positioning requirements of a cleaning task, acquiring an environmental image acquired by the image acquisition device and point cloud data acquired by the lidar; determining an initial pose corresponding to the current environment based on the environmental image; adjusting the initial pose based on the point cloud data to obtain the current pose of the cleaning device; and controlling the cleaning device to perform the cleaning task based on the current pose.

[0007] A second aspect of this application provides a cleaning device, comprising: a body; a cleaning component disposed on the body for cleaning the ground; an image acquisition device for acquiring environmental images; a lidar for acquiring point cloud data; and a control device for performing the positioning method as described in the first aspect of this application.

[0008] In summary, the positioning method and cleaning equipment provided in this application include: responding to the positioning requirements of a cleaning task, acquiring environmental images collected by an image acquisition device and point cloud data collected by a lidar, thereby determining an initial pose corresponding to the current environment based on the environmental images, adjusting the initial pose based on the point cloud data to determine the current pose of the cleaning equipment, and controlling the cleaning equipment to perform the cleaning task based on the current pose. This application uses environmental images from the image acquisition device and point cloud data collected by the lidar to jointly position the cleaning equipment. Compared to existing technologies where the cleaning equipment needs to rotate at least one revolution to collect point cloud data, this application enables the cleaning equipment to determine its initial pose from the environmental images without rotation, and to determine its current pose by adjusting the initial pose based on the limited FoV point cloud data. Therefore, the positioning method provided in this application can more effectively position the cleaning equipment, thereby reducing the energy consumption and time cost required for positioning. Furthermore, since the initial pose of the cleaning equipment can be determined from the environmental images, it is not necessary to control the cleaning equipment to perform a large number of map-based matching attempts, further reducing the computational load, energy consumption, and time cost required for positioning. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram illustrating the application scenario of this application;

[0011] Figure 2 A schematic diagram illustrating a scenario for positioning cleaning equipment;

[0012] Figure 3 A schematic diagram of the structure of an embodiment of the cleaning equipment provided in this application;

[0013] Figure 4 This application provides a schematic diagram of the layout structure of a cleaning device;

[0014] Figure 5 A flowchart illustrating an embodiment of the positioning method provided in this application;

[0015] Figure 6 A schematic diagram of the position determination based on environmental image provided in this application;

[0016] Figure 7A schematic diagram of a measured image used in this application to determine the initial pose based on environmental images;

[0017] Figure 8 A schematic diagram illustrating the determination of the current pose based on point cloud data, provided for this application;

[0018] Figure 9 A schematic diagram illustrating the calculation process for determining the current pose based on point cloud data provided in this application;

[0019] Figure 10 A schematic diagram of a measured radar used to determine the current pose based on point cloud data, provided for this application;

[0020] Figure 11 A schematic diagram illustrating the movement of the cleaning equipment provided in this application;

[0021] Figure 12 A schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] This application is used in cleaning equipment and in scenarios involving the control of cleaning equipment. Figure 1 This is a schematic diagram illustrating the application scenario of this application. Figure 1 In the example shown, the cleaning device 10 is used as an example of the application of a robot vacuum cleaner in a home cleaning scenario. The cleaning device 10 can replace manual labor to complete the cleaning work of the floor through autonomous navigation and adjustment of cleaning strategies.

[0025] In other examples, the cleaning device 10 includes, but is not limited to, robotic vacuum cleaners, robotic floor scrubbers, robotic vacuum and mop combos, robotic lawnmowers, and robotic snowplows. Cleaning tasks may also include floor washing, mopping, sweeping, lawn mowing, and snow removal. The cleaning device 10 can perform cleaning using either a front-sweeping-then-mopping method or a separate sweeping-and-mopping method. The front-sweeping-then-mopping method allows sweeping and mopping simultaneously, improving cleaning efficiency. The separate sweeping-and-mopping method allows sweeping first, followed by mopping, improving cleaning effectiveness.

[0026] In other possible scenarios, the cleaning device 10 can specifically be an autonomous robot capable of moving autonomously within a work area and completing cleaning tasks without external human input or control. The work area can include indoor and outdoor areas. Indoor areas can include family rooms, offices, shopping malls, factory workshops, etc. Outdoor areas can include lawns, gardens, roads, etc.

[0027] Cleaning equipment 10 can connect to server 20 via a network, and terminal device 30 can also connect to server 20 via a network, enabling communication between terminal device 30 and cleaning equipment 10 through server 20. This allows user 40 of cleaning equipment 10 to control cleaning equipment 10 to perform floor cleaning tasks via terminal device 30. Alternatively, in some communication technologies, terminal device 30 can also communicate directly with cleaning equipment 10. This application does not limit the communication method of devices such as cleaning equipment 10 and terminal device 30.

[0028] In order to complete the cleaning task, the cleaning device 10 needs to plan its own path and cleaning strategy. In a specific application scenario, after acquiring the cleaning task, the cleaning device 10 plans its path based on its current location and the map of the target area to be cleaned, and then travels along the planned path to complete the cleaning task of the target area.

[0029] More specifically, the process of creating and using a map by the cleaning device 10 can generally be divided into two stages: the mapping stage and the positioning stage. In the mapping stage, the cleaning device 10 moves within the target area to build a map, and in the positioning stage, the cleaning device 10 determines its own position in real time based on the map.

[0030] During the positioning phase, if the cleaning device 10 drifts due to sensor errors or dynamic changes in the environment, it may be unable to accurately determine its own position in real time based on the map. In this case, the cleaning device 10 can re-determine its position on the map through positioning. This process is also known as re-localization.

[0031] Figure 2 A schematic diagram illustrating a scenario for positioning cleaning equipment, such as... Figure 2 As shown, when the cleaning device 10 obtains a cleaning task at position A1 in the target space, or after obtaining a cleaning task within the station, it dismounts from the station and travels to the designated position A1. The cleaning device 10 first collects point cloud data using a lidar to determine its current location at position A1, and then plans a travel path L1 starting from position A1, thus performing the cleaning task according to path L1. Similarly, when the cleaning device 10 obtains a cleaning task at position A2 in the target space, or after obtaining a cleaning task within the station, it dismounts from the station and travels to the designated position A2, the cleaning device 10 first collects point cloud data using a lidar to determine its current location at position A2, and then plans a travel path L2 starting from position A2, thus performing the cleaning task according to path L2, and so on.

[0032] In existing technologies, the cleaning device 10 primarily uses point cloud data from a LiDAR scanner for positioning. During positioning, due to the limited field of view (FoV) of the LiDAR, the cleaning device 10 needs to rotate at least one full turn to allow it to acquire point cloud data of the current environment. Subsequently, if the initial pose of the cleaning device 10 is known, the current point cloud data can be directly matched with a map, achieving positioning by minimizing the discrepancies between the point cloud and the map. If the initial pose of the cleaning device 10 is unknown, it needs to be controlled to attempt matching between point cloud data from different angles at multiple preset locations on the map, ultimately selecting the pose with the highest matching score as the positioning result.

[0033] As can be seen, in order to complete the positioning, the existing cleaning device 10 needs to rotate at least one revolution to collect point cloud data, which increases the energy consumption and time cost required for positioning. In addition, if there is no initial pose of the cleaning device 10, it is necessary to control the cleaning device 10 to perform a large number of matching attempts based on the map, which further increases the computational load, energy consumption and time cost required for the cleaning device 10 to perform positioning.

[0034] Based on this, this application provides a positioning method and a cleaning device, enabling the cleaning device 10 to perform positioning more effectively, thereby reducing the computational load, energy consumption and time cost required for positioning.

[0035] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0036] Figure 3 A schematic diagram of the structure of an embodiment of the cleaning equipment provided in this application is shown below. Figure 3 The cleaning device 10 shown can be applied to, for example... Figure 1 In the scenario shown, specifically, such as Figure 3 The cleaning device 10 shown includes:

[0037] The control device 100, as the core control unit in the cleaning equipment 10, can integrate and process sensor data, communication data, control commands, etc., and is specifically used for planning cleaning paths and coordinating the work of various components in the cleaning task of the cleaning equipment 10, enabling intelligent decision-making and control of the autonomous operation of the cleaning equipment 10. Specifically, the control device 100 can control other components in the cleaning equipment 10 to perform corresponding operations by executing cleaning methods, thereby controlling the cleaning equipment 10 to perform corresponding cleaning tasks. For example, the control device 100 can be a central processing unit (CPU), a microcontroller unit (MCU), a system on a chip (SoC), or other controllers, or it can be other devices or circuit structures with relevant control functions.

[0038] Cleaning device 101, mounted on the body of cleaning equipment 10, is used for sweeping the floor. (Reference) Figure 1 In the example shown, the body of the cleaning device 10 can be circular, square, or triangular, etc. The cleaning device 101 is mounted on the body and includes one or more of the following: a side brush, a roller brush, a mop, or a vacuum module. The side brush gathers foreign objects and moves them towards the center of the bottom of the cleaning robot. The roller brush sweeps up foreign objects from the bottom of the cleaning robot, allowing them to enter the dust collection box through the suction port. The mop is used for wiping or mopping the floor; the mop can be a disc mop, roller mop, flat mop, tracked mop, etc. The cleaning device 101 is communicatively connected to the control device 100, which can be used to control the cleaning device 101 to perform cleaning tasks. This application does not limit the specific implementation of the cleaning device 101 and its control method.

[0039] Image acquisition device 102 can be used to acquire images of the area in front of the cleaning equipment 10 in its direction of travel. Specifically, the environmental image can be a visible light image. Image acquisition device 102 is communicatively connected to control device 100, which controls the image acquisition device 102 to acquire images. In one embodiment, image acquisition device 102 can be an RGB (Red Green Blue) camera or an RGBD camera. The RGBD camera, combined with height information, enables more accurate identification based on the images acquired.

[0040] In one embodiment, such as Figure 2 The cleaning device 10 shown also includes a supplementary lighting device 103, which can be used to provide supplementary lighting when the image acquisition device 102 acquires images. The supplementary lighting device 103 is communicatively connected to the control device 100, which can be used to control the opening and closing of the supplementary lighting device 103.

[0041] The lidar 104 can be used to collect point cloud data in front of the cleaning equipment 10 in a certain FoV direction. The lidar 104 is communicatively connected to the control device 100, and the control device 100 is used to control the lidar 104 to collect point cloud data.

[0042] For example, Figure 4 This application provides a schematic diagram of the layout structure of a cleaning device, such as... Figure 4 A schematic diagram showing the positions of an image acquisition device 102, a supplementary lighting device 103, and a lidar 104 mounted on the body of a cleaning device 10 is shown. The image acquisition device 102, the supplementary lighting device 103, and the lidar 104 are all mounted on the body of the cleaning device 10 in the forward direction. The image acquisition device 102 can be used to take pictures in the forward direction of the cleaning device 10, the supplementary lighting device 103 can be used to provide supplementary lighting in the forward direction of the cleaning device 10, and the lidar 104 can be used to acquire point cloud data within a certain FoV in the forward direction of the cleaning device 10.

[0043] In another embodiment, the image acquisition device 102, the supplementary lighting device 103, and the lidar 104 may also be located in other positions. The control device 100 can control the image acquisition device 102, the supplementary lighting device 103, and the lidar 104 to rotate, so that the control device 100 can control the image acquisition device 102, the supplementary lighting device 103, and the lidar 104 to rotate to the direction of travel toward the cleaning equipment 10 to acquire images.

[0044] Figure 5 A flowchart illustrating an embodiment of the positioning method provided in this application is shown below. Figure 5 The positioning method shown can be applied to, for example, Figure 3 The cleaning equipment 10 shown is specifically executed by the control device 100 of the cleaning equipment 10. Or, as Figure 5 The positioning method shown can also be applied to other planar motion robots, allowing the robot to perform the positioning. Specifically, such as... Figure 5 The positioning methods shown include:

[0045] S101: In response to the positioning requirements of the cleaning task, the control device 100 acquires the environmental images acquired by the image acquisition device 102 and the point cloud data acquired by the lidar 104.

[0046] In one scenario, after obtaining the preset time for the cleaning task to be performed, the cleaning device 10 can, before starting to perform the cleaning task, first execute actions such as... Figure 5 The positioning method shown locates the current position to determine the current pose of the cleaning device 10, and then plans a travel path based on the current pose to perform the cleaning task.

[0047] In a specific implementation scenario, the cleaning device 10, based on acquiring the cleaning task to be performed, disembarks from the station and travels to the designated location, then performs actions such as... Figure 5 The positioning method shown locates the current position to determine the current pose of the cleaning device 10, and then plans a travel path based on the current pose to perform the cleaning task.

[0048] In another scenario, the cleaning device 10 can repeatedly perform actions at a preset frequency during the cleaning task. Figure 5 The positioning method shown locates the current position, thereby determining the current pose, and then controls the cleaning equipment 10 to continue moving along the planned path more accurately based on the current pose in order to continue performing the cleaning task.

[0049] In a specific implementation scenario, the cleaning device 10 can perform actions such as... while in motion during the cleaning task. Figure 5 The positioning method shown locates the current position; alternatively, during the execution of a cleaning task, the cleaning device 10 can be controlled to switch from a moving state to a stopped state, and in the stopped state, it can perform actions such as... Figure 5 The positioning method shown is used to locate the current position.

[0050] In one embodiment, in response to the positioning requirement of the cleaning task in any of the above scenarios, the control device 100 controls the image acquisition device 102 to acquire environmental images and controls the lidar 104 to acquire point cloud data. In another embodiment, the image acquisition device 102 acquires environmental images at preset intervals and sends them to the control device 100, and the lidar 104 acquires point cloud data at preset intervals and sends it to the control device 100. Then, in response to the positioning requirement of the cleaning task in any of the above scenarios, the control device 100 performs subsequent calculations using the environmental images and point cloud data acquired in S101.

[0051] Understandably, to achieve more accurate positioning later, the image acquisition device 102 and the laser acquisition device 104 acquire environmental images and point cloud data simultaneously. Alternatively, the control device 100 can synchronize the acquired environmental images and point cloud data according to time information and perform subsequent calculations on the environmental images and point cloud data from the same or similar times.

[0052] In one embodiment, when the cleaning device 10 includes a supplementary lighting device 103, the control device 100 can further control the supplementary lighting device 103 to turn on when it determines that the brightness of the environmental image acquired by the image acquisition device 102 is less than a first brightness threshold. This allows the supplementary lighting device 103 to provide supplementary lighting to the front of the cleaning device 10 in its direction of travel, and the control device 100 controls the image acquisition device 102 to acquire environmental images while the supplementary lighting device 103 is on. After the image acquisition device 102 acquires the environmental image, the control device 100 can control the supplementary lighting device 103 to turn off to save energy.

[0053] The brightness of the environmental image can be specifically represented by the brightness (BV) value or the average grayscale value of the image. The control device 100 can determine whether to control the supplementary lighting device 103 to be turned on based on whether the average or maximum brightness of the environmental image previously acquired by the image acquisition device 102 is less than a first brightness threshold. Alternatively, the control device 100 can also control the image acquisition device 102 to acquire a brightness test image, and determine whether to control the supplementary lighting device 103 to be turned on when the image acquisition device 102 actually acquires environmental images thereafter based on whether the average or maximum brightness of the brightness test image is less than the first brightness threshold.

[0054] In another embodiment, the cleaning device 10 is further equipped with a brightness sensor, which can be used to detect the ambient brightness in front of the cleaning device 10 as it moves. If the control device 100 determines through the brightness sensor that the ambient brightness is less than a second brightness threshold, it controls the supplementary lighting device 103 to turn on, providing supplementary lighting to the front of the cleaning device 10 in its direction of travel. The control device 102 then captures an environmental image while the supplementary lighting device 103 is on. The brightness sensor allows for more precise control of the supplementary lighting device 103, effectively improving the quality of the captured environmental image.

[0055] In one embodiment, after receiving the environmental image acquired by the image acquisition device 102, the control device 100 can further preprocess the environmental image to improve its quality, thereby increasing the efficiency of subsequent environmental image processing. The preprocessing of the environmental image includes at least one of brightness enhancement, white balance adjustment, or adaptive exposure processing.

[0056] In another embodiment, when the control device 100 turns on the supplementary lighting device 103 to provide supplementary lighting in front of the cleaning equipment 10 in the direction of travel, and controls the image acquisition device 102 to acquire environmental images while the supplementary lighting device 103 is turned on, the environmental images are then preprocessed. However, when the control device 100 does not turn on the supplementary lighting device 103, preprocessing can be skipped, and the environmental images can be directly processed to reduce the amount of computation and improve processing efficiency.

[0057] In one embodiment, after receiving the point cloud data collected by the lidar 104, the control device 100 can further preprocess the point cloud data to improve its quality, thereby increasing the efficiency of subsequent point cloud data processing. The preprocessing of the point cloud data includes at least one of the following: filtering, noise reduction, enhancement, completion, coordinate system transformation, etc.

[0058] S102: Based on the environmental image acquired in S101, the control device 100 determines the initial pose of the cleaning device 10 corresponding to the current environment.

[0059] Specifically, the control device 100 first determines the initial pose of the cleaning device 10 based on the environmental image. Since this pose is related to the environmental image and cannot completely and accurately indicate the actual pose of the cleaning device, the pose determined based on the environmental image is recorded as the initial pose. The pose includes the current position and orientation of the cleaning device 10.

[0060] In one embodiment, the control device 100 extracts feature information from the environmental image, matches the feature information with the feature information of historical environmental images stored in the database, and determines the loop closure image from the historical environmental images. Then, the pose of the loop closure image at the corresponding moment or the pose at the most recent moment is used as the initial pose of the cleaning device 10.

[0061] For example, Figure 6 This application provides a positional illustration for determining the initial pose based on environmental images, such as... Figure 6 As shown, taking the cleaning device 10 performing a cleaning task within a square target area as an example, the feature object S included in this target area is a sofa. The feature information of the historical environmental images includes historical environmental images corresponding to multiple positions of the cleaning device 10 within the target area in the figure. Assuming the cleaning device 10 is currently at position Q0 and captures an environmental image while performing the positioning method, the control device 100 extracts the feature information of the environmental image and matches it with the feature information of the historical environmental images. If it determines that the feature information of the historical environmental image captured by the cleaning device 10 at position Q1 is closest to the feature information of the environmental image at position Q0, then the control device 100 records the historical environmental image at position Q1 as a loopback image and uses the pose corresponding to this loopback image as the initial pose of the current cleaning device 10.

[0062] In this embodiment, the control device 100 compares the environmental image with the stored historical environmental image using feature information. When the historical environmental image corresponding to the environmental image is determined, it can be considered that the cleaning device 10 has moved to the same or similar position as the historical environmental image, which is equivalent to detecting the "loopback" information of the cleaning device 10. Therefore, in this embodiment, the matched historical environmental image is recorded as the loopback image.

[0063] In one embodiment, the feature information can be information about a feature object, or it can be information about other objects whose pixel values ​​differ significantly from their surrounding pixels. These objects have large pixel gradients in the image, and their features can be matched to determine the loop closure. For the feature information of the historical environment images stored in the plotting device 10, for each feature point in the historical environment image, several pixel positions are selected around it according to a pre-set rule, and the relationship between these pixels and the feature point pixels is recorded, which is the descriptor. Thus, all the feature points of an image and their corresponding descriptors constitute the features of that image.

[0064] For example, Figure 7 This is a schematic diagram of a measured image used in this application to determine the initial pose based on environmental images, as shown below. Figure 7 As shown, assuming the environmental image captured by the cleaning device 10 at position Q0 is Q0-P, this environmental image Q0-P includes a feature object S, which is a door. The historical environmental image captured by the cleaning device 10 at position Q1 is Q1-P, which also includes the same feature object, and the feature information of the two feature objects is most similar. Therefore, the historical environmental image Q1-P is denoted as the loopback image, and the pose corresponding to the loopback image is used as the initial pose of the current cleaning device 10.

[0065] In one embodiment, in S102, if the initial pose of the cleaning device 10 cannot be determined based on the environmental image acquired in S101, the control device 100 controls the cleaning device 10 to rotate by a preset angle, acquires a new environmental image acquired by the image acquisition device 102, and determines the initial pose of the cleaning device 10 based on the new environmental image. Combined with... Figure 6 In the example shown, when the cleaning device 10 does not capture the feature object S at position Q0 and the initial pose of the cleaning device 10 cannot be determined, the control device 100 can control the cleaning device 10 to rotate at position Q0. After the cleaning device 10 rotates, the environmental image re-captured by the image acquisition device 102 includes the feature object S. By comparing the historical environmental images, the loop closure image can be determined, thereby more accurately and effectively determining the initial pose of the cleaning device 10.

[0066] S103: Based on the point cloud data obtained in S101, the control device 100 adjusts the initial pose obtained in S102 to determine the current pose of the cleaning device 10.

[0067] Specifically, since the initial pose determined in S102 cannot completely and accurately indicate the actual pose of the cleaning equipment, the control device 100 optimizes and adjusts the initial pose based on point cloud data to determine the optimized pose. Compared with the initial pose, this pose can more accurately indicate the actual pose of the cleaning equipment. Therefore, the pose adjusted based on point cloud data is recorded as the current pose.

[0068] In one embodiment, the control device 100 converts the point cloud data to a map coordinate system and performs point cloud matching processing based on the point cloud data and the existing map point cloud data in the map coordinate system, thereby optimizing the initial pose to determine the current pose of the cleaning device 10.

[0069] Point cloud matching refers to matching the point cloud data of the latest frame with the map point cloud data if a loop closure is detected based on the environmental image, and calculating the pose transformation between the map point cloud data of the latest frame and the loop closure frame, thereby achieving localization.

[0070] For example, Figure 8 This application provides a positional illustration based on point cloud data for determining the current pose, such as... Figure 8 As shown, the map point cloud data corresponding to position Q1 in the map coordinate system includes point cloud data D1 of feature object S. Based on the point cloud data collected by the cleaning device 10 at the current position Q0, the point cloud data corresponding to feature object S is D0. The control device can determine the pose adjustment method of the current position Q0 relative to position Q1 according to the relative relationship between point cloud data D1 and point cloud data D0. Then, according to the pose adjustment method, the initial pose is adjusted to obtain the current pose of the cleaning device 10.

[0071] In one embodiment, the control device 100 specifically constructs a map search tree based on map point cloud data, thereby fitting the point cloud data with the nearest neighbor points in the map search tree to form multiple straight lines, and using a nonlinear optimization framework to determine the pose adjustment method based on the pose change that minimizes the sum of multiple straight lines.

[0072] For example, Figure 9 This application provides a schematic diagram of the calculation process for determining the current pose based on point cloud data, as shown below. Figure 9The control device 100 shown can determine each feature point d01, d02, ... in the point cloud data, and determine the nearest neighbor feature point corresponding to each feature point and fit it to form a straight line. For example, the nearest neighbor of feature point d01 in the map point cloud data is d11, and the fitted straight line is K1; the nearest neighbor of feature point d02 in the map point cloud data is d12, and the fitted straight line is K2, and so on, eventually forming multiple straight lines K1, K2, ... The control device 100 adjusts the position of the point cloud data D0 to minimize the sum of the above multiple straight lines K1, K2, ..., and determines the current pose based on the current position of the point cloud data D0.

[0073] For example, Figure 10 The actual radar diagram provided for determining the current pose based on point cloud data in this application is as follows: Figure 10 As shown, the point cloud data D1 in the map data includes point cloud data of feature object 1 as D1-1 and point cloud data of feature object 2 as D1-2. In the point cloud data D0 collected at the current position Q0, the point cloud data of feature object 1 is D0-1 and the point cloud data of feature object 2 is D0-2. It can be seen that there is a deviation between point cloud data D0 and point cloud data D1. By adjusting the overall rotation angle and position of point cloud data D0, the relative relationship between point cloud data D0 and point cloud data D1 can be determined, thereby determining the pose adjustment method between the initial pose and the current pose. Then, the current pose can be obtained by adjusting the initial pose based on the pose adjustment method, making point cloud data D0 and point cloud data D1 in the measured radar image closer, and ideally, making them coincide.

[0074] S103: The control device 100 controls the cleaning equipment to perform a cleaning task based on the current pose determined in S102.

[0075] In one embodiment, after the control device 100 determines the current pose, it can determine the current position and orientation of the cleaning device 10. The control device 100 can plan a travel route according to the acquired cleaning task, control the cleaning device 10 to travel in the planned direction and start to perform the cleaning task, or control the cleaning device 10 to switch from the stopped state to the continued travel state and travel in the planned direction to start to perform the cleaning task.

[0076] In another embodiment, after determining the current pose, the control device 100 determines the current position and orientation of the cleaning device 10, thereby updating the inaccurate positioning. This allows the control device 100 to control the cleaning device 10 to continue moving from the current position to continue performing the cleaning task based on the more accurate current pose, or to control the cleaning device 10 to switch from a stopped state to a continuing moving state and move in the planned direction to begin performing the cleaning task.

[0077] For example, Figure 11 A schematic diagram illustrating the movement of the cleaning equipment provided in this application, as shown below. Figure 11 As shown, after the cleaning device 10 stops at point Q0 and performs the positioning method to determine the current pose, it can be controlled to start moving from point Q0 according to the current pose and the travel path L0, so as to start or continue to perform the cleaning task.

[0078] In summary, the positioning method provided in this application, in response to the positioning needs of the cleaning equipment, acquires environmental images collected by the image acquisition device 102 and point cloud data collected by the lidar 104, thereby determining the initial pose corresponding to the current environment based on the environmental images, adjusting the initial pose based on the point cloud data, determining the current pose of the cleaning equipment 10, and finally controlling the cleaning equipment 10 to perform cleaning tasks based on the current pose.

[0079] As can be seen, this application uses environmental images from the image acquisition device 102 and point cloud data collected by the lidar 104 to jointly locate the cleaning device 10. Compared with the prior art, where the cleaning device 10 needs to rotate at least one revolution to collect point cloud data during positioning, this application can determine the initial pose from the environmental image without rotation, and then adjust the initial pose based on the point cloud data with a limited FoV to determine the current pose of the cleaning device 10. This enables the cleaning device 10 with the lidar 104 having a limited FoV to achieve accurate and efficient positioning without rotation.

[0080] Therefore, the positioning method provided in this application can more effectively locate the cleaning device 10, thereby reducing the energy consumption and time cost required for positioning. Furthermore, since the initial pose of the cleaning device 10 can be determined through environmental images, it is not necessary to control the cleaning device 10 to perform a large number of matching attempts based on the map, which further reduces the computational load, energy consumption and time cost required for the cleaning device 10 to be positioned.

[0081] Furthermore, in the above embodiments of this application, a method for locating a cleaning device 10 is provided. Before executing the above-mentioned positioning method, the control device 100 of the cleaning device 10 can also acquire the feature information and corresponding pose of historical environmental images in advance and store them in a database, so as to determine the initial pose of the cleaning device 10 by using the feature information and corresponding pose of the historical environmental images stored in the database when executing the positioning method.

[0082] In one embodiment, the cleaning device 10 can acquire feature information and corresponding pose of historical environmental images sent by other devices.

[0083] In another embodiment, the cleaning device 10 can generate feature information and corresponding poses of historical environmental images by collecting environmental images and point cloud data. The cleaning device 10 can determine the feature information of historical environmental quantities during the mapping process, thereby recording the information of several key poses in the indoor environment during the mapping stage, which can be used as a reference for determining the initial pose during subsequent loop closure detection.

[0084] Specifically, the control device 100 of the cleaning equipment 10 can determine the feature information of the historical environmental image through the following steps:

[0085] S201: Control the cleaning equipment 10 to acquire historical environmental images collected by the image acquisition device 102, historical point cloud data collected by the lidar 14, and the pose corresponding to the historical environmental images while it is in motion. The timestamps of these three types of data correspond one-to-one.

[0086] S202: Based on historical environmental images, determine the feature information of the historical environmental images.

[0087] In one embodiment, in S202, the control device 100 specifically determines the feature information of the historical environment image through the following steps:

[0088] S2020: The control device 100 performs conversion processing on the environmental image. The environmental image directly acquired by the image sensor, such as the image acquisition device 102, is generally an RGB color image, while subsequent processing requires conversion to grayscale. Therefore, the control device 100 can convert the color environmental image into a grayscale environmental image, at which point a pixel location changes from 3-channel data to single-channel data.

[0089] S2021: The control device 100 performs quality detection on historical environmental images and historical point cloud data to retain historical environmental images and historical point cloud data whose quality meets preset conditions.

[0090] In one embodiment, the control device 100 performs quality detection on the historical environment image by uniformly dividing the historical environment image into several grids with a side length of 'a'. Each grid is a superpixel on the historical environment image. If the width of the historical environment image is 'w' and the height is 'h', then the number of superpixels is 'h*w / a^2'. For each superpixel, the average gray value g_ave of all pixels within its range is calculated, and then its pixel variance g_cov is calculated. If g_cov is greater than a threshold of 1, it indicates that the superpixel contains sufficient texture information and is considered a valid superpixel. After the calculation is completed, the ratio of the number of valid superpixels to the total number of superpixels is calculated. If it is greater than a threshold of 2, the quality of the historical environment image is considered to meet the preset conditions; if it is less than or equal to a threshold of 2, the quality of the historical environment image is considered not to meet the preset conditions. Since the success of determining the initial pose based on image loop closure detection depends on the presence of rich features in the environmental image, if the cleaning device 10 is too close to the wall, or if it is dark, or if there is rotational blurring, the information contained in the acquired environmental image is insufficient. Therefore, the environmental image cannot effectively find the correct visual loop closure relationship. Thus, by performing quality inspection on the historical environment, historical environmental images that do not meet the preset conditions can be effectively filtered out, ensuring that the subsequent loop closure detection based on the environmental image and the historical environmental image can more accurately and effectively determine the initial pose.

[0091] In one embodiment, the quality detection of point cloud data by the control device 100 includes: calculating parameters of the historical point cloud data, including the number of point cloud points ptNum, which is the number of laser points contained in the historical point cloud data of the current frame; the average distance disMean, which is the average distance between all points in the frame and the lidar 104; the median distance disMed, which is the median distance value after sorting the distances between all points in the frame and the lidar 104; the variance of the distance disCov, which is the variance of the distance between all points in the frame and the lidar 104; the variance in the x-direction xCov, which is the variance of all points in the frame in the x-direction; and the variance in the y-direction yCov, which is the variance of all points in the frame in the y-direction. If the above parameters of the point cloud data are greater than thresholds 3 to 8 respectively, and xCov / yCov and yCov / xCov are within the range of thresholds 9 and 10, then the quality of the historical point cloud data is considered to meet the preset conditions; otherwise, the quality of the historical point cloud data is considered not to meet the preset conditions. In particular, when the cleaning equipment 10 is in certain degraded scenarios, especially with limited FoV, the quality of the point cloud data acquired by the LiDAR 104 may be abnormal. Point cloud data with abnormal quality may result in inaccurate real-time positioning results. As historical point cloud data, if the loopback frame obtained after loopback detection of the latest point cloud data in subsequent processes is the same, and its pose is inaccurate, then the pose used for positioning will also be inaccurate. Therefore, quality detection of historical point cloud data is used to ensure that the pose of the frames used to build the database is correct.

[0092] As can be seen, by selecting higher quality historical environmental images and historical point cloud data, this embodiment can form more accurate and effective feature information of historical environmental images, enabling the control device 100 to more effectively determine the initial pose based on the feature information of historical environmental images, thereby more accurately, effectively and quickly locating the cleaning equipment 10, making the positioning process more seamless and improving the user experience of the cleaning equipment 10.

[0093] S2022: Use the pose search tree to perform nearest neighbor search to determine keyframes in historical environment images.

[0094] A keyframe is a collection of all the data required at a specific moment, and the information between different keyframes has certain differences. For example, if the cleaning device 10 is stationary and continuously receives several frames of data with little difference, these data will not constitute a keyframe. In the embodiments of this application, a keyframe includes a timestamp, the pose at that moment, the point cloud data of that moment, and the environmental image at that moment.

[0095] In one embodiment, to ensure sufficient difference between the data added to the database to control the database size, the control device 100 determines in S2022 whether the latest frame is a keyframe. For example, the control device 100 can make the determination based on their pose relationships. For the latest frame data, its pose includes the x-coordinate, y-coordinate, and yaw angle. Using x and y, a nearest neighbor search is performed in the pose search tree with a radius r. If there are no other keyframe pose points within the range, or although there are other keyframe poses, the difference between its pose and the yaw of all keyframes within the range is greater than the threshold 11, then the latest frame is considered a keyframe, and its pose is added to the pose search tree, recording its frame number and pose. The pose search tree is used for keyframe determination; it records the poses of historical keyframes. When a new frame arrives, the new frame pose is compared with the poses in the tree. If there are no similar poses, the latest frame is considered a keyframe.

[0096] S2023: Extract feature information of key frames in historical environmental images using extraction algorithms.

[0097] In this process, the control device 100 extracts feature information from the keyframes determined in S2022. Specifically, this extraction can be achieved using traditional algorithms such as FAST, ORB, Shi-Tomas, SIFT, and SURF, or deep learning-based algorithms such as Superpoint, to extract the feature information of the keyframes, resulting in a set of pixels, including their pixel coordinates and pixel values.

[0098] S203: Associate and store the feature information of key frames in historical environment images, the point cloud data corresponding to the key frames, and the poses corresponding to historical environment images in the database.

[0099] Specifically, in order to describe an environmental image through feature information, there may be subtle differences between environmental images with loop relationships and historical environmental images, so it is impossible to determine loops by directly comparing the two images; however, they have similar features, so feature extraction is a prerequisite for finding loop relationships.

[0100] In one embodiment, the control device 100 acquires the descriptor of each feature point in the current historical environment image in S2024. Currently, there are various descriptor extraction methods, such as traditional extraction algorithms like BRIEF, ORB, BRISK, SIFT, and SURF, or deep learning-based extraction algorithms like superpoint. All of these descriptor extraction methods are supported here. Once all feature points and corresponding descriptors of the image have been extracted, they are combined to obtain the image features of the current image.

[0101] In one specific implementation, the control device 100 recursively clusters the feature information of key frames in the historical environment image through a tree-like hierarchical structure, adds it to the nodes of each level of the multi-level feature vocabulary, forms a bag-of-words for the feature information of the historical environment image, and stores it in the database.

[0102] The bag-of-words is used to store a database of image features. Visual loop closure detection compares the feature information of the latest environmental image with the feature information of the historical environmental images stored in the bag-of-words to find loop closure relationships. The database contains the bag-of-words, the pose search tree, and the pose and frame number of each keyframe.

[0103] In one embodiment, the control device 100 adds image features to the bag-of-words and uses the DBoW bag-of-words tool to pass the image features and image numbers into the bag-of-words, which is a tree structure. After passing in the image features, the relevant features are automatically clustered and placed in the corresponding leaf nodes.

[0104] In summary, through the above steps, the control device 100 of the cleaning equipment 10 determines the feature information of the historical environment image based on the collected historical environmental images and historical point cloud data. This allows the control device 100 to perform subsequent actions such as... Figure 5 The positioning method shown is used when positioning.

[0105] For example, when the control device 100 performs actions such as Figure 5 In the positioning method shown, after acquiring environmental images and point cloud data, the acquired environmental images and point cloud data can be subjected to quality inspection and processing in the same manner as in S2020-S2021, and subsequent calculations can be performed on environmental images and point cloud data that meet preset conditions. Subsequently, the control device 100 can determine the point cloud data of the environmental image in the same manner as in S2022-S2023.

[0106] In one embodiment, the control device 100 matches the feature information of the environmental image with the feature information of the historical environmental image to retrieve the sequence number of the loop closure frame in the historical environmental image from the bag-of-words database, and obtains the loop closure score s. If the score is greater than a threshold of 12, it is considered that the visual loop closure relationship is satisfied. The corresponding loop closure frame pose Tml is found according to the loop closure frame ID as the initial pose for localization. Here, the subscript m of Tml is the map coordinate system, l is the loop closure frame coordinate system, and Tml is the pose of the loop closure frame in the map coordinate system.

[0107] In one embodiment, the control device 100 performs point cloud matching on point cloud data based on an initial pose, specifically including:

[0108] 1021: Construct a map search tree using map points. Extract all points from the map, which are a series of 2D coordinates (x, y), and input them into the search tree to find the correspondence between the current frame and map points.

[0109] 1022: The point cloud data is transformed to the map coordinate system corresponding to the LiDAR 104 based on the initial pose. First, the point cloud data is transformed to the map coordinate system according to the TML. Then, using a map search tree, the nearest N map points are found for each point in the point cloud data. Since the indoor environment is highly structured, the area around each map point is basically a straight line. Therefore, a 2D straight line can be fitted using these N points: A*x+B*y+C=0. For each point (xl, yl) in the frame, the distance from the point to the straight line is obtained as d=A*xl+B*yl+C. By optimizing the distance from each point to the corresponding nearest straight line and minimizing it, the current pose Tmf of the point cloud data in the map coordinate system of the current frame can be obtained, where f is the coordinate system of the current frame.

[0110] In one embodiment, after the control device 100 calculates the current pose Tmf using the above method, it further transforms the point cloud data of the current frame into a map coordinate system using Tmf. Using a map search tree, it finds the nearest N map points for each point in the point cloud data in the tree, and then fits a 2D straight line using these N points and calculates the corresponding distance d. The average of all d values ​​is calculated. If it is less than a threshold of 13, the current pose Tmf is considered correctly calculated, and the localization is successful; otherwise, the current pose Tmf is considered incorrectly calculated, and localization can be performed again.

[0111] In one embodiment, to prevent positioning error detection during the mapping phase from causing inaccurate current pose, the control device 100, by executing, such as Figure 5 After obtaining the current pose, the localization method shown transforms the point cloud data to the map coordinate system based on the current pose. Then, based on the relative relationship between the point cloud data and the map point cloud data in the map coordinate system, it determines whether the current pose is valid.

[0112] Specifically, for each data point in the point cloud data, the control device 100 follows the... Figure 9 In the same way shown, the nearest neighbor map point is determined from the map search tree built based on map point cloud data, and each data point in the point cloud data is fitted with its corresponding map point to form multiple straight lines, K1, K2, etc. If the average length of the multiple straight lines meets the valid condition, the current pose is determined to be valid; if the average length of the multiple straight lines does not meet the valid condition, the current pose is determined to be invalid.

[0113] Furthermore, if the difference between the current pose and the SLAM or other positioning results of the current cleaning device 10 is greater than a preset difference value, it is considered that the positioning in the mapping and cleaning / positioning stages is incorrect. The current pose obtained from the positioning is used to replace the positioning result to achieve positioning correction, thereby continuing to control the cleaning device 10 to perform the cleaning task based on the current pose of the positioning.

[0114] It can be seen that the above embodiments lack error detection, especially means of positioning recovery after error detection, in the mapping and positioning stages. This application, during the cleaning task performed by the cleaning equipment 10, can promptly recover from positioning errors by executing measures such as... Figure 5 The loop closure detection and positioning method shown is used for positioning, and the positioning result is compared with the latest positioning result. Error detection and positioning recovery can be achieved seamlessly. Once a positioning error is detected, the current positioning pose can be automatically used as the current positioning to continue to perform cleaning tasks more accurately and effectively. At the same time, it also improves the intelligence level and user experience of the cleaning equipment 10.

[0115] This application also provides a cleaning device, comprising: a body; a cleaning component disposed on the body for cleaning the ground; an image acquisition device for acquiring environmental images; a lidar for acquiring point cloud data; and a control device for executing any of the aforementioned positioning methods of this application.

[0116] In the foregoing embodiments of this application, the positioning method provided by the embodiments of this application has been described. In order to realize the functions of the methods provided by the embodiments of this application, the control device 100, as the execution subject, can implement the above functions through hardware structure and / or software modules. Whether a certain function is executed by hardware structure, software module, or hardware structure plus software module depends on the specific application and design constraints of the technical solution.

[0117] It should be understood that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a module can be a separately established processing element, or it can be integrated into a chip within the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0118] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together to implement a system-on-a-chip (SOC).

[0119] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0120] For example, Figure 12 A schematic diagram of the structure of an embodiment of the electronic device provided in this application is shown below. Figure 12 The electronic device 2000 shown can be used to perform the positioning method provided in any embodiment of this application.

[0121] In one embodiment, such as Figure 12 The control device 2000 shown includes one or more processors 2001 and a memory 2002. The memory 2002 stores computer-executable instructions, and the processor 2001 can execute the computer-executable instructions stored in the memory 2002. When the computer-executable instructions are executed by the processor 2001, the processor 2001 implements the positioning method provided in any of the foregoing embodiments of this application.

[0122] In one embodiment, such as Figure 12 The control device 2000 shown also includes a communication interface 2003, through which the processor 2001 can communicate with other devices, such as sending and receiving data through the communication interface 2003.

[0123] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0124] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0125] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0126] This application also provides a chip for executing instructions, which is used to execute the positioning method provided in any of the foregoing embodiments of this application.

[0127] This application also provides a computer program product, including a computer program that, when executed, implements the positioning method provided in any of the foregoing embodiments of this application.

[0128] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, can be used to implement the positioning method provided in any of the foregoing embodiments of this application.

[0129] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0130] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0131] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0133] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0134] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A positioning method, characterized in that, Applied to cleaning equipment, the cleaning equipment including an image acquisition device and a lidar, the method includes: In response to the positioning requirements of the cleaning task, the system acquires environmental images collected by the image acquisition device and point cloud data collected by the lidar. Based on the environmental image, determine the initial pose corresponding to the current environment; The initial pose is adjusted based on the point cloud data to obtain the current pose of the cleaning equipment; Based on the current pose, the cleaning device is controlled to perform the cleaning task.

2. The positioning method according to claim 1, characterized in that, The location requirement in response to a cleaning task includes at least one of the following: The cleaning device performs positioning after obtaining a preset time for the cleaning task; In response to the cleaning equipment being acquired, the equipment dismounts from the station and travels to a designated location for positioning. In response to the cleaning device performing the cleaning task, positioning is repeated at a preset frequency.

3. The positioning method according to claim 1, characterized in that, Also includes: If the initial pose of the cleaning equipment cannot be determined based on the environmental image, the cleaning equipment is controlled to rotate by a preset angle, and the environmental image acquired by the image acquisition device is reacquired. Alternatively, if the initial pose of the cleaning device cannot be determined based on the environmental image, the cleaning device is controlled to rotate in a preset direction, and the environmental image acquired by the image acquisition device is reacquired. Alternatively, if the initial pose of the cleaning device cannot be determined based on the environmental image, the cleaning device can be controlled to rotate in a preset direction by a preset angle, and the environmental image acquired by the image acquisition device can be reacquired.

4. The positioning method according to any one of claims 1-3, characterized in that, Determining the initial pose corresponding to the current environment based on the environmental image includes: Extract feature information from the environmental image; The feature information is matched with the historical feature information of historical environmental images stored in the database to determine the loop closure image from the historical environmental images, and the pose corresponding to the loop closure image is used as the initial pose of the cleaning device.

5. The positioning method according to claim 4, characterized in that, The step of adjusting the initial pose based on the point cloud data to obtain the current pose of the cleaning device includes: Transform the point cloud data to the map coordinate system according to the initial pose; In the map coordinate system, the pose adjustment method is determined based on the relative relationship between the point cloud data and the map point cloud data; Based on the aforementioned pose adjustment method, the initial pose is adjusted to obtain the current pose of the cleaning equipment.

6. The positioning method according to claim 4, characterized in that, The method for determining pose adjustment based on the relative relationship between the point cloud data and the map point cloud data includes: For each data point in the point cloud data, the nearest neighbor map point is determined from the map search tree constructed based on the map point cloud data; The point cloud data is fitted to its corresponding map point to form multiple straight lines; The pose adjustment method is determined based on the pose change that minimizes the sum of the multiple straight lines.

7. The positioning method according to any one of claims 4-6, characterized in that, After obtaining the current pose of the cleaning device, the method further includes: Transform the point cloud data to the map coordinate system according to the current pose; In the map coordinate system, the validity of the current pose is determined based on the relative relationship between the point cloud data and the map point cloud data.

8. The positioning method according to claim 7, characterized in that, Determining whether the current pose is valid based on the relative relationship between the point cloud data and the map point cloud data includes: For each data point in the point cloud data, the nearest neighbor map point is determined from the map search tree constructed based on the map point cloud data; The point cloud data is fitted to its corresponding map point to form multiple straight lines; If the average length of the multiple straight lines meets the valid condition, then the current pose is determined to be valid.

9. The positioning method according to any one of claims 4-8, characterized in that, Also includes: The system acquires historical environmental images acquired by the image acquisition device, historical point cloud data acquired by the lidar, and poses corresponding to the historical environmental images. The historical environment images and historical point cloud data are subjected to quality detection, and the historical environment images and historical point cloud data that meet the preset quality conditions are retained; Nearest neighbor search is performed using a pose search tree to determine keyframes in the historical environment image; Feature information of key frames in the historical environment image is extracted using an extraction algorithm; The feature information of key frames in the historical environment image, as well as the point cloud data and historical poses corresponding to the key frames, are associated and stored in the database.

10. A cleaning device, characterized in that, include: body; A cleaning component, mounted on the machine body, is used for cleaning the floor; Image acquisition device, used to acquire environmental images; LiDAR is used to collect point cloud data; A control device for performing the positioning method as described in any one of claims 1-9.

Images