Robots, repositioning methods, apparatus and readable storage media thereof

Abstract

This invention provides a robot and its relocalization method, apparatus, and readable storage medium. The robot relocalization method includes: upon receiving a relocalization request, acquiring a first image output by an image acquisition device, the first image including an image of the robot's environment; determining the robot's target pose information based at least partially on the first image; and controlling a lidar to perform relocalization based on the target pose information. In this process, the impact of obstacles in dynamic environments on lidar relocalization is reduced, eliminating limitations in handling dynamic environments and real-time performance.

Description

Technical Field

[0001] This invention relates to the field of control technology, and more specifically, to a robot and its repositioning method, apparatus, and readable storage medium. Background Technology

[0002] With the development of artificial intelligence technology and the continuous improvement of market demand, higher requirements are being placed on the functions and performance of robotic vacuum cleaners.

[0003] In the relevant technical solutions, the robotic vacuum cleaners on the market mainly rely on LiDAR for mapping and positioning in terms of navigation and positioning. Although LiDAR technology has high accuracy, LiDAR positioning is easily affected by obstacles in dynamic environments, and has limitations in handling dynamic environments and real-time performance. Summary of the Invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art or related technologies, such as the susceptibility of lidar positioning to obstacles in dynamic environments, and the limitations in handling dynamic environments and real-time performance.

[0005] Therefore, a first aspect of the present invention is to provide a method for repositioning a robot.

[0006] A second aspect of the present invention is that a robot repositioning device is provided.

[0007] A third aspect of the invention is that it provides another repositioning device for a robot.

[0008] A fourth aspect of the present invention is that a readable storage medium is provided.

[0009] A fifth aspect of the invention is that a robot is provided.

[0010] In view of this, according to a first aspect of the present invention, the present invention provides a robot relocalization method, the robot including an image acquisition device and a lidar, the robot relocalization method comprising: upon receiving a relocalization request, acquiring a first image output by the image acquisition device, the first image including an image of the environment in which the robot is located; determining target pose information of the robot based at least in part on the first image; and controlling the lidar to perform relocalization based on the target pose information.

[0011] This invention proposes a robot relocation method. By running the above-mentioned robot relocation method, upon receiving a relocation request, the target pose information of the robot can be determined based on the first image output by the image acquisition device. Then, the lidar is controlled to perform relocation based on the target pose information. In this process, vision can be used to assist the lidar in relocation in a dynamic environment, thereby reducing the impact of obstacles on lidar relocation in a dynamic environment and eliminating the limitations in handling dynamic environments and real-time performance.

[0012] The above technical solution is adaptable and robust to dynamic obstacles, dynamic blurred scenes, and changes in lighting. It combines a multi-sensor fusion relocation method with image acquisition devices and lidar, which increases the amount of information and improves the accuracy of robot localization and relocation.

[0013] Furthermore, when responding to a relocation request, the lidar performs relocation based on the target pose information. Compared to global relocation, this can narrow the relocation range and thus improve the relocation efficiency.

[0014] In the above technical solution, the robot's existing camera can be reused. Obviously, in dynamic environments, vision can be used to assist the lidar in repositioning, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments. This eliminates the limitations in handling dynamic environments and real-time performance, while also reducing the robot's manufacturing cost.

[0015] In addition, the robot relocation method proposed in this invention has the following additional technical features.

[0016] In some technical solutions, optionally, the target pose information of the robot is determined at least partially based on the first image, specifically including: inputting the first image into a scene recognition model to obtain first environmental information and a first feature vector output by the scene recognition model; filtering target data information from a target database based on the first environmental information; obtaining the target similarity between the target feature vector in the target data information and the first feature vector; and using the pose information in the target data information as the target pose information if the target similarity is greater than a similarity threshold.

[0017] In this technical solution, during the process of determining the target pose information, a scene recognition model can be used to process the first image, thereby identifying the first environment information and the first feature vector corresponding to the first image. The first environment information is used to filter target data information from the target database. The target similarity between the target feature vector and the first feature vector in the filtered target data information is calculated. Based on the comparison result of the target similarity and the similarity threshold, it is determined whether to use the pose information in the target data information as the target pose information.

[0018] This process can provide accurate pose information for lidar repositioning, thereby improving the repositioning efficiency.

[0019] In the above technical solution, the first feature vector can be understood as the vector obtained by the scene recognition model encoding the first image.

[0020] In some technical solutions, optionally, each piece of data information in the target database includes the robot's pose information, the environmental information of the robot's environment, and feature vectors. Target data information is obtained by filtering from the target database based on the first environmental information. Specifically, this includes: comparing the environmental information in each piece of data information with the first environmental information; and using the data information corresponding to the target environmental information in the target database as the target data information, where the target environmental information is the same as the first environmental information.

[0021] In this technical solution, a target database is pre-built so that after processing the first image using a scene recognition model to obtain the first environmental information and the first feature vector, the target data information whose environmental information is the first environmental information can be retrieved from the target database using the first environmental information.

[0022] In this process, the target data information is the data information in the target database. Therefore, the data information that is closest to the first image can be found by searching the target database. Thus, when responding to the relocation request, the lidar performs relocation based on the target pose information in the target data information. Compared with global relocation, the positioning range can be narrowed, thereby improving the positioning efficiency of relocation.

[0023] In some technical solutions, the environmental information in each data item may optionally include at least one of the following: room type, floor material.

[0024] In this technical solution, when environmental information includes room type, target data information can be filtered based on room type. During this process, the scene in which the robot is located can be identified, and target data information can be filtered based on the identified scene, thereby improving the accuracy of target data information.

[0025] In some technical solutions, the room type can be one or more of the following: kitchen, bathroom, bedroom, and living room.

[0026] In the above technical solution, the floor material can reflect the ground information of the scene in which the robot is located. When the environmental information includes the ground material, the floor material can be used to filter the target data information, thereby improving the accuracy of the target data information.

[0027] In the above technical solution, room type and floor material can be used simultaneously to filter target data information. During this process, data information that is most similar to the first image can be found from the target database for repositioning by the lidar.

[0028] In some technical solutions, optionally, when there are multiple target data information, the robot relocalization method further includes: determining a first similarity and a second similarity among multiple target similarities, where the first similarity is the maximum similarity among multiple target similarities, and the second similarity is the maximum similarity among target similarities excluding the first similarity; and, based on the case where the target similarity is greater than a similarity threshold, using the pose information in the target data information as the target pose information, specifically including: when the first similarity and the second similarity are greater than the similarity threshold, obtaining the distance value between the first pose information and the second pose information, where the first pose information is the pose information corresponding to the first similarity, and the second pose information is the pose information corresponding to the second similarity; and, when the distance value is less than the distance threshold, using the first pose information as the target pose information.

[0029] In this technical solution, when multiple target data information is selected, the similarity of multiple targets is sorted to determine the maximum similarity and the second maximum similarity among the multiple target similarities, which are the first similarity and the second similarity in this application.

[0030] After determining the first similarity and the second similarity, the similarity between the first feature vector and the feature vector in the target data information is judged by comparing the first similarity and the second similarity with the similarity threshold respectively. If both the first similarity and the second similarity are greater than the similarity threshold, the target data information is considered to be the filtered data information that is similar to the first image and can be used as a reference when the lidar is repositioned.

[0031] By calculating the distance between the first pose information and the second pose information, it is possible to determine whether the pose information corresponding to the two selected target data information is the same pose information or whether they are similar pose information. If the obtained distance value is less than the distance threshold, the first pose information and the second pose information are considered to be two relatively close pose information. Therefore, the first pose information corresponding to the first similarity with higher similarity can be used as the reference point when the lidar is repositioned.

[0032] Therefore, when responding to a relocation request, the lidar performs relocation based on the first pose information. Compared with global relocation, this can reduce the positioning range and thus improve the positioning efficiency.

[0033] In the above technical solution, the best pose information can be selected to guide the lidar for repositioning, thereby improving the positioning efficiency of repositioning.

[0034] In some technical solutions, the robot relocalization method may optionally include: controlling the lidar to perform global relocalization when the target similarity is less than or equal to a similarity threshold.

[0035] In this technical solution, if the target similarity is less than or equal to the similarity threshold, it is considered that the filtered target data information does not match the first image. At this time, the target data information obtained by visual recommendation is no longer referred to, but relocation is performed directly in the global scope.

[0036] In this process, if the selected target data does not match the first image, the target data can be ignored and global relocation can be used for positioning. This can eliminate the impact of erroneous target data on lidar relocation while ensuring positioning accuracy.

[0037] In some technical solutions, the robot relocalization method may optionally include: acquiring a second image output by an image acquisition device and pose information corresponding to the second image output by the robot's pose sensor while the robot is performing a task; storing the pose information corresponding to the second image in a target list; acquiring a third image output by an image acquisition device and pose information corresponding to the third image output by the pose sensor; storing the pose information corresponding to the third image in the target list if the distance between the pose information corresponding to the second image and the pose information corresponding to the third image is greater than a set distance; processing the image corresponding to each pose information in the target list using a scene recognition model to obtain environmental information and feature vectors of the robot's environment; and constructing a target database based on each pose information in the target list, the environmental information of the robot's environment, and the feature vectors.

[0038] In this technical solution, the target database can be automatically built during the execution of the task. Obviously, there is no need to execute the process of building the target database separately, which reduces the amount of data to be processed and the amount of computation for data processing.

[0039] The above technical solution enables the use of visual relocation methods to narrow down the LiDAR positioning from a global scope to a local scope, thereby improving the efficiency of relocation.

[0040] Furthermore, in the above technical solution, feature vectors can be extracted during the execution of the task, reducing the utilization of image processor resources and saving hardware resources.

[0041] In the above technical solution, before storing the pose information corresponding to the third image in the target list, the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is calculated, and the calculated distance is compared with a preset distance. The pose information corresponding to the third image is stored in the target list only when the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is greater than the preset distance.

[0042] During this process, two pose information that are relatively far apart can be filtered out and stored in the target list, thereby establishing a target database containing different pose information.

[0043] In some technical solutions, the robot relocalization method may optionally include: acquiring sample images, which are images acquired by image acquisition devices under different working conditions; classifying each sample image to determine the room type and floor material corresponding to each sample image, thereby obtaining training samples; training the network model using a preset network model of the training samples to obtain a trained network model; and performing model conversion on the trained network model to obtain a scene recognition model.

[0044] In this technical solution, images are acquired by an image acquisition device under different working conditions in order to construct training samples covering different working conditions. In the process of classifying each sample image, each image can be cleaned and classified with multiple labels in order to determine the room type and floor material corresponding to each sample image.

[0045] The preset network model is trained using the above training samples to obtain the trained network model. The model is then transformed to obtain a scene recognition model that conforms to this technical solution.

[0046] According to a second aspect of the present invention, the present invention provides a robot repositioning device. The robot includes an image acquisition device and a lidar. The robot repositioning device includes: an acquisition unit, configured to acquire a first image output by the image acquisition device upon receiving a repositioning request, the first image including an image of the environment in which the robot is located; a determination unit, configured to determine target pose information of the robot based at least in part on the first image; and a positioning unit, configured to control the lidar to perform repositioning based on the target pose information.

[0047] This invention proposes a robot repositioning device that, upon receiving a repositioning request, determines the robot's target pose information based on a first image output by an image acquisition device, and then controls the lidar to perform repositioning based on the target pose information. In this process, vision can be used to assist the lidar in repositioning in dynamic environments, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments and eliminating limitations in handling dynamic environments and real-time performance.

[0048] The above technical solution is adaptable and robust to dynamic obstacles, dynamic blurred scenes, and changes in lighting. It combines a multi-sensor fusion relocation method with image acquisition devices and lidar, which increases the amount of information and improves the accuracy of robot localization and relocation.

[0049] Furthermore, when responding to a relocation request, the lidar performs relocation based on the target pose information. Compared to global relocation, this can narrow the relocation range and thus improve the relocation efficiency.

[0050] In the above technical solution, the robot's existing camera can be reused. Obviously, in dynamic environments, vision can be used to assist the lidar in repositioning, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments. This eliminates the limitations in handling dynamic environments and real-time performance, while also reducing the robot's manufacturing cost.

[0051] In addition, the robot repositioning device proposed in this invention has the following additional technical features.

[0052] In some technical solutions, the determining unit is specifically used for: inputting a first image into a scene recognition model to obtain first environmental information and a first feature vector output by the scene recognition model; filtering target data information from a target database based on the first environmental information; obtaining the target similarity between the target feature vector in the target data information and the first feature vector; and, if the target similarity is greater than a similarity threshold, using the pose information in the target data information as the target pose information.

[0053] In this technical solution, during the process of determining the target pose information, a scene recognition model can be used to process the first image, thereby identifying the first environment information and the first feature vector corresponding to the first image. The first environment information is used to filter target data information from the target database. The target similarity between the target feature vector and the first feature vector in the filtered target data information is calculated. Based on the comparison result of the target similarity and the similarity threshold, it is determined whether to use the pose information in the target data information as the target pose information.

[0054] This process can provide accurate pose information for lidar repositioning, thereby improving the repositioning efficiency.

[0055] In the above technical solution, the first feature vector can be understood as the vector obtained by the scene recognition model encoding the first image.

[0056] In some technical solutions, optionally, each data information in the target database includes the robot's pose information, the environmental information of the robot's environment, and a feature vector. The determining unit is specifically used to: compare the environmental information in each data information with the first environmental information; and take the data information corresponding to the target environmental information in the target database as the target data information, wherein the target environmental information is the same as the first environmental information.

[0057] In this technical solution, a target database is pre-built so that after processing the first image using a scene recognition model to obtain the first environmental information and the first feature vector, the target data information whose environmental information is the first environmental information can be retrieved from the target database using the first environmental information.

[0058] In this process, the target data information is the data information in the target database. Therefore, the data information that is closest to the first image can be found by searching the target database. Thus, when responding to the relocation request, the lidar performs relocation based on the target pose information in the target data information. Compared with global relocation, the positioning range can be narrowed, thereby improving the positioning efficiency of relocation.

[0059] In some technical solutions, the environmental information in each data item may optionally include at least one of the following: room type, floor material.

[0060] In this technical solution, when environmental information includes room type, target data information can be filtered based on room type. During this process, the scene in which the robot is located can be identified, and target data information can be filtered based on the identified scene, thereby improving the accuracy of target data information.

[0061] In some technical solutions, the room type can be one or more of the following: kitchen, bathroom, bedroom, and living room.

[0062] In the above technical solution, the floor material can reflect the ground information of the scene in which the robot is located. When the environmental information includes the ground material, the floor material can be used to filter the target data information, thereby improving the accuracy of the target data information.

[0063] In the above technical solution, room type and floor material can be used simultaneously to filter target data information. During this process, data information that is most similar to the first image can be found from the target database for repositioning by the lidar.

[0064] In some technical solutions, optionally, when there are multiple target data information, a determining unit is specifically used to: determine a first similarity and a second similarity among multiple target similarities, where the first similarity is the maximum similarity among multiple target similarities, and the second similarity is the maximum similarity among target similarities excluding the first similarity; if the first similarity and the second similarity are greater than a similarity threshold, obtain the distance value between the first pose information and the second pose information, where the first pose information is the pose information corresponding to the first similarity, and the second pose information is the pose information corresponding to the second similarity; if the distance value is less than a distance threshold, use the first pose information as the target pose information.

[0065] In this technical solution, when multiple target data information is selected, the similarity of multiple targets is sorted to determine the maximum similarity and the second maximum similarity among the multiple target similarities, which are the first similarity and the second similarity in this application.

[0066] After determining the first similarity and the second similarity, the similarity between the first feature vector and the feature vector in the target data information is judged by comparing the first similarity and the second similarity with the similarity threshold respectively. If both the first similarity and the second similarity are greater than the similarity threshold, the target data information is considered to be the filtered data information that is similar to the first image and can be used as a reference when the lidar is repositioned.

[0067] By calculating the distance between the first pose information and the second pose information, it is possible to determine whether the pose information corresponding to the two selected target data information is the same pose information or whether they are similar pose information. If the obtained distance value is less than the distance threshold, the first pose information and the second pose information are considered to be two relatively close pose information. Therefore, the first pose information corresponding to the first similarity with higher similarity can be used as the reference point when the lidar is repositioned.

[0068] Therefore, when responding to a relocation request, the lidar performs relocation based on the first pose information. Compared with global relocation, this can reduce the positioning range and thus improve the positioning efficiency.

[0069] In the above technical solution, the best pose information can be selected to guide the lidar for repositioning, thereby improving the positioning efficiency of repositioning.

[0070] In some technical solutions, the positioning unit may optionally also be used to: control the lidar to perform global repositioning when the target similarity is less than or equal to a similarity threshold.

[0071] In this technical solution, if the target similarity is less than or equal to the similarity threshold, it is considered that the filtered target data information does not match the first image. At this time, the target data information obtained by visual recommendation is no longer referred to, but relocation is performed directly in the global scope.

[0072] In this process, if the selected target data does not match the first image, the target data can be ignored and global relocation can be used for positioning. This can eliminate the impact of erroneous target data on lidar relocation while ensuring positioning accuracy.

[0073] In some technical solutions, optionally, the determining unit is further configured to: acquire a second image output by an image acquisition device and pose information corresponding to the second image output by the robot's pose sensor when the robot is performing a task; store the pose information corresponding to the second image in a target list; acquire a third image output by an image acquisition device and pose information corresponding to the third image output by a pose sensor; if the distance between the pose information corresponding to the second image and the pose information corresponding to the third image is greater than a set distance, store the pose information corresponding to the third image in the target list; process the image corresponding to each pose information in the target list using a scene recognition model to obtain environmental information and feature vectors of the robot's environment; and construct a target database based on each pose information in the target list, the environmental information of the robot's environment, and the feature vectors.

[0074] In this technical solution, the target database can be automatically built during the execution of the task. Obviously, there is no need to execute the process of building the target database separately, which reduces the amount of data to be processed and the amount of computation for data processing.

[0075] The above technical solution enables the use of visual relocation methods to narrow down the LiDAR positioning from a global scope to a local scope, thereby improving the efficiency of relocation.

[0076] Furthermore, in the above technical solution, feature vectors can be extracted during the execution of the task, reducing the utilization of image processor resources and saving hardware resources.

[0077] In the above technical solution, before storing the pose information corresponding to the third image in the target list, the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is calculated, and the calculated distance is compared with a preset distance. The pose information corresponding to the third image is stored in the target list only when the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is greater than the preset distance.

[0078] During this process, two pose information that are relatively far apart can be filtered out and stored in the target list, thereby establishing a target database containing different pose information.

[0079] In some technical solutions, optionally, the determining unit is also used for: acquiring sample images, which are images acquired by image acquisition devices under different working conditions; classifying each sample image to determine the room type and floor material corresponding to each sample image, thereby obtaining training samples; training the network model using a preset network model of the training samples to obtain a trained network model; and performing model conversion on the trained network model to obtain a scene recognition model.

[0080] In this technical solution, images are acquired by an image acquisition device under different working conditions in order to construct training samples covering different working conditions. In the process of classifying each sample image, each image can be cleaned and classified with multiple labels in order to determine the room type and floor material corresponding to each sample image.

[0081] The preset network model is trained using the above training samples to obtain the trained network model. The model is then transformed to obtain a scene recognition model that conforms to this technical solution.

[0082] According to a third aspect of the present invention, the present invention provides a robot relocation apparatus, including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the robot relocation method as described above.

[0083] According to a fourth aspect of the present invention, the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the robot relocation method as described above.

[0084] According to a fifth aspect of the present invention, the present invention provides a robot comprising: a repositioning device as described in any of the above-described robots; and / or a readable storage medium as described above.

[0085] In some technical solutions, the robot may optionally include a robotic vacuum cleaner.

[0086] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0087] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0088] Figure 1 A flowchart illustrating a robot relocation method according to an embodiment of the present invention is shown.

[0089] Figure 2 A schematic diagram of the structure of a robot according to an embodiment of the present invention is shown:

[0090] Figure 3 A schematic block diagram of a robot relocation device according to an embodiment of the present invention is shown:

[0091] Figure 4 A schematic block diagram of another robot relocation device according to an embodiment of the present invention is shown:

[0092] Figure 5 This invention illustrates a schematic diagram of the principle of robot relocation in an embodiment of the present invention:

[0093] Figure 6 A schematic diagram of the structure of a model output in an embodiment of the present invention is shown;

[0094] Figure 7 A schematic block diagram of another robot repositioning device according to an embodiment of the present invention is shown.

[0095] in, Figure 2 The correspondence between the reference numerals and component names in the attached drawings is as follows:

[0096] 200 robots, 202 image acquisition devices, and 204 lidar. Detailed Implementation

[0097] To better understand the above aspects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0098] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0099] In one embodiment of this application, such as Figure 1 and Figure 2 As shown, a robot relocalization method is provided. The robot 200 includes an image acquisition device 202 and a lidar 204. The robot relocalization method includes:

[0100] Step 102: Upon receiving a relocation request, acquire the first image output by the image acquisition device. The first image includes an image of the environment in which the robot is located.

[0101] Step 104: Determine the target pose information of the robot, at least in part, based on the first image;

[0102] Step 106: Based on the target pose information, control the lidar to perform repositioning.

[0103] This invention proposes a robot relocation method. By running the above-mentioned robot relocation method, upon receiving a relocation request, the target pose information of the robot can be determined based on the first image output by the image acquisition device. Then, the lidar is controlled to perform relocation based on the target pose information. In this process, vision can be used to assist the lidar in relocation in a dynamic environment, thereby reducing the impact of obstacles on lidar relocation in a dynamic environment and eliminating the limitations in handling dynamic environments and real-time performance.

[0104] In the above embodiments, the robot is adaptable and robust to dynamic obstacles, dynamic blurred scenes, and changes in lighting. The multi-sensor fusion relocalization method, which combines image acquisition devices and lidar, increases the amount of information and improves the accuracy of robot localization and relocalization.

[0105] Furthermore, when responding to a relocation request, the lidar performs relocation based on the target pose information. Compared to global relocation, this can narrow the relocation range and thus improve the relocation efficiency.

[0106] In the above embodiments, the robot's existing cameras can be reused. Obviously, in dynamic environments, vision can be used to assist the lidar in repositioning, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments. This eliminates the limitations in handling dynamic environments and real-time performance, while also reducing the robot's manufacturing cost.

[0107] In some embodiments, optionally, the target pose information of the robot is determined at least in part based on the first image, specifically including: inputting the first image into a scene recognition model to obtain first environmental information and a first feature vector output by the scene recognition model; filtering target data information from a target database based on the first environmental information; obtaining the target similarity between the target feature vector in the target data information and the first feature vector; and using the pose information in the target data information as the target pose information if the target similarity is greater than a similarity threshold.

[0108] In this embodiment, during the process of determining the target pose information, the first image can be processed using a scene recognition model to identify the first environment information and the first feature vector corresponding to the first image. The first environment information is used to filter target data information from the target database. The target similarity between the target feature vector and the first feature vector in the filtered target data information is calculated. The pose information in the target data information is used as the target pose information based on the comparison result between the target similarity and the similarity threshold.

[0109] This process can provide accurate pose information for lidar repositioning, thereby improving the repositioning efficiency.

[0110] In the above embodiments, the first feature vector can be understood as the vector obtained by the scene recognition model encoding the first image.

[0111] In some embodiments, optionally, each piece of data information in the target database includes the robot's pose information, the environmental information of the robot's environment, and a feature vector. Target data information is obtained by filtering from the target database based on the first environmental information, specifically including: comparing the environmental information in each piece of data information with the first environmental information; and using the data information corresponding to the target environmental information in the target database as the target data information, wherein the target environmental information is the same as the first environmental information.

[0112] In this embodiment, a target database is pre-built so that after processing the first image using a scene recognition model to obtain the first environmental information and the first feature vector, the target data information whose environmental information is the first environmental information can be retrieved from the target database using the first environmental information.

[0113] In this process, the target data information is the data information in the target database. Therefore, the data information that is closest to the first image can be found by searching the target database. Thus, when responding to the relocation request, the lidar performs relocation based on the target pose information in the target data information. Compared with global relocation, the positioning range can be narrowed, thereby improving the positioning efficiency of relocation.

[0114] In some embodiments, the environmental information in each piece of data may optionally include at least one of the following: room type, floor material.

[0115] In this embodiment, when the environmental information includes room type, the target data information can be filtered according to the room type. In this process, the scene in which the robot is located can be identified, and the target data information can be filtered according to the identified scene, thereby improving the accuracy of the target data information.

[0116] In some embodiments, the room type can be one or more of the following: kitchen, bathroom, bedroom, and living room.

[0117] In the above embodiments, the floor material can reflect the ground information of the scene in which the robot is located. When the environmental information includes the ground material, the floor material can be used to filter target data information, thereby improving the accuracy of the target data information.

[0118] In the above embodiments, room type and floor material can be used simultaneously to filter target data information. During this process, data information that is most similar to the first image can be found from the target database for repositioning by the lidar.

[0119] In some embodiments, optionally, when there are multiple target data information, the robot relocalization method further includes: determining a first similarity and a second similarity among multiple target similarities, wherein the first similarity is the maximum similarity among multiple target similarities, and the second similarity is the maximum similarity among target similarities excluding the first similarity; and, based on the case where the target similarity is greater than a similarity threshold, using the pose information in the target data information as the target pose information, specifically including: when the first similarity and the second similarity are greater than the similarity threshold, obtaining the distance value between the first pose information and the second pose information, wherein the first pose information is the pose information corresponding to the first similarity, and the second pose information is the pose information corresponding to the second similarity; and, when the distance value is less than the distance threshold, using the first pose information as the target pose information.

[0120] In this embodiment, when multiple target data information is selected, the similarity of the multiple targets is sorted to determine the maximum similarity and the second maximum similarity among the multiple target similarities, which are the first similarity and the second similarity in this application.

[0121] After determining the first similarity and the second similarity, the similarity between the first feature vector and the feature vector in the target data information is judged by comparing the first similarity and the second similarity with the similarity threshold respectively. If both the first similarity and the second similarity are greater than the similarity threshold, the target data information is considered to be the filtered data information that is similar to the first image and can be used as a reference when the lidar is repositioned.

[0122] By calculating the distance between the first pose information and the second pose information, it is possible to determine whether the pose information corresponding to the two selected target data information is the same pose information or whether they are similar pose information. If the obtained distance value is less than the distance threshold, the first pose information and the second pose information are considered to be two relatively close pose information. Therefore, the first pose information corresponding to the first similarity with higher similarity can be used as the reference point when the lidar is repositioned.

[0123] Therefore, when responding to a relocation request, the lidar performs relocation based on the first pose information. Compared with global relocation, this can reduce the positioning range and thus improve the positioning efficiency.

[0124] In the above embodiments, the optimal pose information can be selected to guide the lidar in repositioning, thereby improving the repositioning efficiency.

[0125] In some embodiments, the robot relocalization method may optionally further include: controlling the lidar to perform global relocalization based on the target similarity being less than or equal to a similarity threshold.

[0126] In this embodiment, if the target similarity is less than or equal to the similarity threshold, it is considered that the filtered target data information does not match the first image. At this time, the target data information obtained by visual recommendation is no longer referred to, but relocation is performed directly in the global scope.

[0127] In this process, if the selected target data does not match the first image, the target data can be ignored and global relocation can be used for positioning. This can eliminate the impact of erroneous target data on lidar relocation while ensuring positioning accuracy.

[0128] In some embodiments, the robot relocalization method may optionally further include: acquiring a second image output by an image acquisition device and pose information corresponding to the second image output by the robot's pose sensor when the robot is performing a task; storing the pose information corresponding to the second image in a target list; acquiring a third image output by an image acquisition device and pose information corresponding to the third image output by a pose sensor; storing the pose information corresponding to the third image in a target list when the distance between the pose information corresponding to the second image and the pose information corresponding to the third image is greater than a set distance; processing the image corresponding to each pose information in the target list using a scene recognition model to obtain environmental information and feature vectors of the robot's environment; and constructing a target database based on each pose information in the target list, the environmental information of the robot's environment, and the feature vectors.

[0129] In this embodiment, the target database can be automatically built during the execution of the task. Obviously, there is no need to execute the process of building the target database separately, which reduces the amount of data to be processed and the amount of computation involved in data processing.

[0130] In the above embodiments, the relocation method using vision can be used to narrow down the LiDAR positioning from a global range to a local range, thereby improving the efficiency of relocation.

[0131] Furthermore, in the above embodiments, feature vectors can be extracted during task execution, reducing the utilization of image processor resources and saving hardware resources.

[0132] In the above embodiments, before storing the pose information corresponding to the third image in the target list, the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is calculated, and the calculated distance is compared with a preset distance. The pose information corresponding to the third image is stored in the target list only when the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is greater than the preset distance.

[0133] During this process, two pose information that are relatively far apart can be filtered out and stored in the target list, thereby establishing a target database containing different pose information.

[0134] In some embodiments, the robot relocalization method may optionally further include: acquiring sample images, which are images acquired by an image acquisition device under different working conditions; classifying each sample image to determine the room type and floor material corresponding to each sample image, thereby obtaining training samples; training the network model using a preset network model of the training samples to obtain a trained network model; and performing model conversion on the trained network model to obtain a scene recognition model.

[0135] In this embodiment, images acquired by the image acquisition device under different working conditions are obtained in order to construct training samples covering different working conditions. In the process of classifying each sample image and performing inverse classification, each image can be cleaned and classified with multiple labels in order to determine the room type and floor material corresponding to each sample image.

[0136] The preset network model is trained using the above training samples to obtain the trained network model. The model is then transformed to obtain a scene recognition model that conforms to this embodiment.

[0137] In one embodiment, such as Figure 3 As shown, the present invention provides a robot repositioning device 300. The robot includes an image acquisition device and a lidar. The robot repositioning device includes: an acquisition unit 302, used to acquire a first image output by the image acquisition device upon receiving a repositioning request. The first image includes an image of the robot's environment; a determination unit 304, used to determine the robot's target pose information based at least in part on the first image; and a positioning unit 306, used to control the lidar to perform repositioning based on the target pose information.

[0138] This invention proposes a robot repositioning device 300, which can determine the robot's target pose information based on the first image output by the image acquisition device when a repositioning request is received, and then control the lidar to perform repositioning based on the target pose information. In this process, vision can be used to assist the lidar in repositioning in dynamic environments, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments and eliminating the limitations in handling dynamic environments and real-time performance.

[0139] In the above embodiments, the robot is adaptable and robust to dynamic obstacles, dynamic blurred scenes, and changes in lighting. The multi-sensor fusion relocalization method, which combines image acquisition devices and lidar, increases the amount of information and improves the accuracy of robot localization and relocalization.

[0140] Furthermore, when responding to a relocation request, the lidar performs relocation based on the target pose information. Compared to global relocation, this can narrow the relocation range and thus improve the relocation efficiency.

[0141] In the above embodiments, the robot's existing cameras can be reused. Obviously, in dynamic environments, vision can be used to assist the lidar in repositioning, thereby reducing the impact of obstacles on lidar repositioning in dynamic environments. This eliminates the limitations in handling dynamic environments and real-time performance, while also reducing the robot's manufacturing cost.

[0142] In some embodiments, the determining unit is specifically configured to: input a first image into a scene recognition model to obtain first environmental information and a first feature vector output by the scene recognition model; filter target data information from a target database based on the first environmental information; obtain the target similarity between the target feature vector in the target data information and the first feature vector; and, if the target similarity is greater than a similarity threshold, use the pose information in the target data information as the target pose information.

[0143] In this embodiment, during the process of determining the target pose information, the first image can be processed using a scene recognition model to identify the first environment information and the first feature vector corresponding to the first image. The first environment information is used to filter target data information from the target database. The target similarity between the target feature vector and the first feature vector in the filtered target data information is calculated. The pose information in the target data information is used as the target pose information based on the comparison result between the target similarity and the similarity threshold.

[0144] This process can provide accurate pose information for lidar repositioning, thereby improving the repositioning efficiency.

[0145] In the above embodiments, the first feature vector can be understood as the vector obtained by the scene recognition model encoding the first image.

[0146] In some embodiments, optionally, each data information in the target database includes robot pose information, environmental information of the robot's environment, and feature vectors. The determining unit 304 is specifically used to: compare the environmental information in each data information with the first environmental information; and take the data information corresponding to the target environmental information in the target database as the target data information, wherein the target environmental information is the same as the first environmental information.

[0147] In this embodiment, a target database is pre-built so that after processing the first image using a scene recognition model to obtain the first environmental information and the first feature vector, the target data information whose environmental information is the first environmental information can be retrieved from the target database using the first environmental information.

[0148] In this process, the target data information is the data information in the target database. Therefore, the data information that is closest to the first image can be found by searching the target database. Thus, when responding to the relocation request, the lidar performs relocation based on the target pose information in the target data information. Compared with global relocation, the positioning range can be narrowed, thereby improving the positioning efficiency of relocation.

[0149] In some embodiments, the environmental information in each piece of data may optionally include at least one of the following: room type, floor material.

[0150] In this embodiment, when the environmental information includes room type, the target data information can be filtered according to the room type. In this process, the scene in which the robot is located can be identified, and the target data information can be filtered according to the identified scene, thereby improving the accuracy of the target data information.

[0151] In some embodiments, the room type can be one or more of the following: kitchen, bathroom, bedroom, and living room.

[0152] In the above embodiments, the floor material can reflect the ground information of the scene in which the robot is located. When the environmental information includes the ground material, the floor material can be used to filter target data information, thereby improving the accuracy of the target data information.

[0153] In the above embodiments, room type and floor material can be used simultaneously to filter target data information. During this process, data information that is most similar to the first image can be found from the target database for repositioning by the lidar.

[0154] In some embodiments, optionally, when there are multiple target data information, the determining unit 304 is specifically used to: determine a first similarity and a second similarity among multiple target similarities, wherein the first similarity is the maximum similarity among multiple target similarities, and the second similarity is the maximum similarity among target similarities other than the first similarity; if the first similarity and the second similarity are greater than a similarity threshold, obtain a distance value between the first pose information and the second pose information, wherein the first pose information is the pose information corresponding to the first similarity, and the second pose information is the pose information corresponding to the second similarity; if the distance value is less than a distance threshold, use the first pose information as the target pose information.

[0155] In this embodiment, when multiple target data information is selected, the similarity of the multiple targets is sorted to determine the maximum similarity and the second maximum similarity among the multiple target similarities, which are the first similarity and the second similarity in this application.

[0156] After determining the first similarity and the second similarity, the similarity between the first feature vector and the feature vector in the target data information is judged by comparing the first similarity and the second similarity with the similarity threshold respectively. If both the first similarity and the second similarity are greater than the similarity threshold, the target data information is considered to be the filtered data information that is similar to the first image and can be used as a reference when the lidar is repositioned.

[0157] By calculating the distance between the first pose information and the second pose information, it is possible to determine whether the pose information corresponding to the two selected target data information is the same pose information or whether they are similar pose information. If the obtained distance value is less than the distance threshold, the first pose information and the second pose information are considered to be two relatively close pose information. Therefore, the first pose information corresponding to the first similarity with higher similarity can be used as the reference point when the lidar is repositioned.

[0158] Therefore, when responding to a relocation request, the lidar performs relocation based on the first pose information. Compared with global relocation, this can reduce the positioning range and thus improve the positioning efficiency.

[0159] In the above embodiments, the optimal pose information can be selected to guide the lidar in repositioning, thereby improving the repositioning efficiency.

[0160] In some embodiments, optionally, the positioning unit 306 is further configured to: control the lidar to perform global repositioning if the target similarity is less than or equal to a similarity threshold.

[0161] In this embodiment, if the target similarity is less than or equal to the similarity threshold, it is considered that the filtered target data information does not match the first image. At this time, the target data information obtained by visual recommendation is no longer referred to, but relocation is performed directly in the global scope.

[0162] In this process, if the selected target data does not match the first image, the target data can be ignored and global relocation can be used for positioning. This can eliminate the impact of erroneous target data on lidar relocation while ensuring positioning accuracy.

[0163] In some embodiments, optionally, the determining unit 304 is further configured to: acquire a second image output by an image acquisition device and pose information corresponding to the second image output by the robot's pose sensor when the robot is performing a task; store the pose information corresponding to the second image in a target list; acquire a third image output by an image acquisition device and pose information corresponding to the third image output by a pose sensor; if the distance between the pose information corresponding to the second image and the pose information corresponding to the third image is greater than a set distance, store the pose information corresponding to the third image in the target list; process the image corresponding to each pose information in the target list using a scene recognition model to obtain environmental information and feature vectors of the robot's environment; and construct a target database based on each pose information in the target list, the environmental information of the robot's environment, and the feature vectors.

[0164] In this embodiment, the target database can be automatically built during the execution of the task. Obviously, there is no need to execute the process of building the target database separately, which reduces the amount of data to be processed and the amount of computation involved in data processing.

[0165] In the above embodiments, the relocation method using vision can be used to narrow down the LiDAR positioning from a global range to a local range, thereby improving the efficiency of relocation.

[0166] Furthermore, in the above embodiments, feature vectors can be extracted during task execution, reducing the utilization of image processor resources and saving hardware resources.

[0167] In the above embodiments, before storing the pose information corresponding to the third image in the target list, the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is calculated, and the calculated distance is compared with a preset distance. The pose information corresponding to the third image is stored in the target list only when the distance between the pose information corresponding to the third image and the pose information corresponding to the second image is greater than the preset distance.

[0168] During this process, two pose information that are relatively far apart can be filtered out and stored in the target list, thereby establishing a target database containing different pose information.

[0169] In some embodiments, optionally, the determining unit 304 is further configured to: acquire sample images, wherein the sample images are images acquired by the image acquisition device under different working conditions; classify each sample image to determine the room type and floor material corresponding to each sample image, thereby obtaining training samples; train the network model using a preset network model of the training samples to obtain a trained network model; and perform model conversion on the trained network model to obtain a scene recognition model.

[0170] In this embodiment, images acquired by the image acquisition device under different working conditions are obtained in order to construct training samples covering different working conditions. In the process of classifying each sample image and performing inverse classification, each image can be cleaned and classified with multiple labels in order to determine the room type and floor material corresponding to each sample image.

[0171] The preset network model is trained using the above training samples to obtain the trained network model. The model is then transformed to obtain a scene recognition model that conforms to this embodiment.

[0172] In one embodiment, the robot is a robotic vacuum cleaner, such as... Figure 4 and Figure 5As shown, the robot's relocalization device 400 includes: a home scene recognition model construction module 402, a pose acquisition and image sampling module 404, a feature vector database construction module 406, and a relocalization module 408.

[0173] Among them, the home scene recognition model construction module 402 involves data collection, classification, model selection and adjustment, model training and parameter adjustment, model conversion, etc., and ultimately achieves the deployment of a home scene recognition model that can run in real time on the robot vacuum cleaner.

[0174] Specifically, the home scene recognition model construction module 402 includes the following steps:

[0175] Data collection: A camera is mounted on the robot vacuum cleaner to collect RGB images. The camera is about 6cm high. The collected images should cover different house types, different times (day / night), different room types, and different floor materials as much as possible, and the scene should match the actual living home environment.

[0176] An RGB image is an image composed of red (R), green (G), and blue (B).

[0177] Data Classification: Based on the images obtained from data collection, data cleaning and multi-label classification are performed. Specifically, home scenes can be divided into 8 room types, such as bathroom, bedroom, dining room, kitchen, living room, balcony, corridor, and entrance hall, and 3 flooring materials, such as tile, wood flooring, and carpet / mat. Multiple labels are allowed for a single image, such as multiple room or flooring materials appearing simultaneously from a certain perspective. Images without any labels are also allowed, such as dark scenes, reflections, or overexposure.

[0178] Model selection: Select a neural network model suitable for the scene recognition task. This step requires comprehensive consideration of on-board computing power, memory, and model accuracy to select the appropriate structure, such as Mobilenetv2.

[0179] Mobilenetv2 is a deep learning image classification network architecture.

[0180] Model training and parameter tuning: The model is trained on a well-classified multi-label dataset using the PyTorch framework, and the parameters are tuned through cross-validation to obtain optimal performance.

[0181] Among them, the PyTorch framework is an open-source deep learning framework for machine learning and deep learning.

[0182] Model Conversion: To facilitate inference on edge devices, the trained PTH format model needs to be converted to ONNX format. ONNX contains two outputs: one is an output containing room type and floor material (dimensions 1, 11), and the other is a feature vector (dimensions 1, 1280) after removing fully connected layers. The structure of the model output is as follows: Figure 6 As shown, it still needs to be further converted into the format required for board-side inference.

[0183] like Figure 6 As shown, GlobalAveragePool represents global average pooling, Flatten represents a flattened layer, Shape represents reshaping, Gather indices=0 represents index assignment, Unsqueeze represents inserting a dimension of size 1 in the specified dimension, Concat represents concatenation, and Reshape represents reshaping.

[0184] Among them, Gemm B<11×1280>C <11> This indicates that the output dimension is limited to 1×11, where output represents the output and features represent the feature vectors.

[0185] ONNX (Open Neural Network Exchange) is an open file format designed for machine learning.

[0186] The pose acquisition and image sampling module 404 is used to acquire the pose information (x, y) of the sweeping robot and the angle threshold θ to sample the image. During the robot cleaning process, the amount of calculation is reduced, while ensuring that the sampling points basically cover every area of ​​the map.

[0187] Specifically, the pose acquisition and image sampling module 404 includes the following steps:

[0188] Pose information acquisition: Obtain pose information from robot sensors, including x, y coordinates and pitch angle θ.

[0189] Image Sampling: During normal machine cleaning, the input image set is traversed, and images are filtered based on given distance and angle thresholds. First, an empty list is created to store visited poses. Then, during iteration, the pose of each image is compared with the list of visited poses. If the distance and angle between the current image's pose and all poses in the visited list are greater than the given thresholds, the image is added to the visited list; this image is a sample. This process is repeated until completion, thus obtaining sampled images with different poses.

[0190] The feature vector database construction module 406 involves using a home scene recognition model as a feature extractor to extract feature vectors, and saving room type, floor material, feature vectors, and pose information in NPY file format as a feature vector database.

[0191] Among them, the NPY file format is a binary file format used by the NumPy library.

[0192] Specifically, the feature vector database construction module 406 includes the following steps:

[0193] Feature extraction: During the cleaning process of the robot vacuum cleaner, the home scene recognition model is called on the sampled samples that conform to the image sampling strategy, and finally the room type, floor material category and feature vector are output.

[0194] The feature vector database is saved as follows: Room type, floor material, feature vectors, and pose information are saved in npy file format to the feature vector database features.npy of the sampled samples. To match dynamic environmental changes, features.npy is rewritten each time the cleaning task restarts. In addition, to reduce local memory consumption, this invention uses the fp16 data type when saving the feature vector database.

[0195] fp16 is a half-precision floating-point number.

[0196] The relocalization module 408 involves similarity calculation and ranking, scene matching, and function implementation. Combining the room type and floor material recognition results, it calculates the cosine similarity of the feature vectors and ranks them based on similarity, finding the optimal relocalization pose under certain conditions.

[0197] Similarity Calculation and Ranking: During the relocalization process, firstly, the robot vacuum needs to acquire the query image and call the model to output the room type, floor material category, and feature vector of the query image; secondly, combining the room type and floor material information returned by the query image, the possible area where the robot is currently located is determined; finally, in the sampled feature vector database, only the cases where the room type, floor material, and query image are the same are considered, and the query feature vector F1 = [a1, a2, ..., a] is calculated sequentially within this range. 1280 [] (that is, the first feature vector in this application) and the sample feature vector F n =[b1,b2,...,b 1280 The cosine similarity of each feature vector in the target database (i.e., the feature vector contained in each data information) is calculated and ranked according to similarity. The more similar two feature vectors are, the closer the cosine similarity is to 1. The formula for calculating cosine similarity is as follows:

[0198]

[0199] Scene matching: To avoid misidentification during visual relocalization in cases of multiple maps or insufficient sampling, a cosine similarity threshold of 0.95 and an Euclidean distance threshold of 2 are set. The poses corresponding to the top two most similar sampling points with a cosine similarity greater than 0.95 are found. The Euclidean distance between the top two poses (x1, y1) and (x2, y2) is calculated. If the Euclidean distance is less than 2, the visual relocalization result is considered reliable, and the pose of the top one most similar sampling point is returned. The formula for calculating the Euclidean distance is as follows:

[0200]

[0201] Where D is the distance between the first pose information and the second pose information, (x1,y1) is the first pose information, and (x2,y2) is the second pose information.

[0202] Functionality: When the robot vacuum initiates a relocation request, it performs local relocation near the pose (x,y) returned by scene matching using LiDAR. If scene matching does not find a pose that meets the conditions, it performs global relocation using LiDAR.

[0203] In this embodiment, the home scene recognition model integrates room classification, floor material recognition, and visual relocation. On the one hand, the results of room classification and floor recognition can be used for map partitioning, defining cleaning order, and improving user experience. On the other hand, feature vectors are extracted when the robot vacuum cleaner performs cleaning work, reducing the use of image processor resources and saving hardware resources.

[0204] In one embodiment, such as Figure 7 As shown, the present invention provides a robot relocation device 700, including a processor 702 and a memory 704. The memory 704 stores programs or instructions that can be executed on the processor 702. When the program or instructions are executed by the processor 702, they implement the steps of the robot relocation method as described above.

[0205] The memory 704 can be used to store software programs and various data. The memory 704 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area can store the operating system, application programs or instructions required for at least one function (such as sound playback function, image playback function, etc.). Furthermore, the memory 704 can include volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0206] In one embodiment, the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the robot relocation method as described above.

[0207] In one embodiment, the present invention provides a robot including: a repositioning device as described in any of the robots described above; and / or a readable storage medium as described above.

[0208] In some embodiments, the robot may optionally include a robotic vacuum cleaner.

[0209] The terms "first" and "second" in the specification and claims of this application may explicitly or implicitly include one or more of the features. In the textual description of this invention, unless otherwise stated, "a plurality of" means two or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0210] In the textual description of this invention, it is understood that, unless explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0211] In the claims, description, and accompanying drawings of this invention, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In the claims, description, and accompanying drawings of this invention, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0212] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for relocalizing a robot, characterized in that, The robot includes an image acquisition device and a lidar, and the robot's relocalization method includes: Upon receiving a relocation request, the robot acquires a first image output by the image acquisition device, the first image including an image of the environment in which the robot is located; The target pose information of the robot is determined based at least in part on the first image; Based on the target pose information, the lidar is controlled to perform repositioning.

2. The method of repositioning a robot according to claim 1, wherein, Determining the target pose information of the robot, at least in part based on the first image, specifically includes: The first image is input into the scene recognition model to obtain the first environmental information and the first feature vector output by the scene recognition model; Based on the first environmental information, target data information is obtained by filtering from the target database; Obtain the target similarity between the target feature vector in the target data information and the first feature vector; If the target similarity is greater than the similarity threshold, the pose information in the target data information is taken as the target pose information.

3. The method of relocating a robot according to claim 2, wherein, Each piece of data information in the target database includes the robot's pose information, the environmental information of the robot's environment, and a feature vector. The step of filtering target data information from the target database based on the first environmental information specifically includes: Compare the environmental information in each of the data information with the first environmental information; The data information corresponding to the target environment information in the target database is used as the target data information, and the target environment information is the same as the first environment information.

4. The method of repositioning a robot according to claim 3, wherein, The environmental information in each of the aforementioned data information includes at least one of the following: Room type, floor material.

5. The method of repositioning a robot of claim 2, wherein, When there are multiple target data information, the robot relocalization method further includes: A first similarity and a second similarity are determined among a plurality of target similarities, wherein the first similarity is the maximum similarity among the plurality of target similarities, and the second similarity is the maximum similarity among the target similarities excluding the first similarity; When the target similarity is greater than a similarity threshold, the pose information in the target data information is used as the target pose information, specifically including: If the first similarity and the second similarity are greater than the similarity threshold, the distance between the first pose information and the second pose information is obtained, where the first pose information is the pose information corresponding to the first similarity and the second pose information is the pose information corresponding to the second similarity. If the distance value is less than the distance threshold, the first pose information is used as the target pose information.

6. The method of repositioning a robot of claim 2, wherein, The robot relocation method further includes: If the target similarity is less than or equal to the similarity threshold, the lidar is controlled to perform global relocation.

7. The method according to any one of claims 2 to 6, wherein, The robot relocation method further includes: When the robot is performing a task, the second image output by the image acquisition device and the pose information corresponding to the second image output by the robot's pose sensor are acquired. Store the pose information corresponding to the second image in the target list; Acquire the third image output by the image acquisition device and the pose information corresponding to the third image output by the pose sensor; If the distance between the pose information corresponding to the second image and the pose information corresponding to the third image is greater than a set distance, the pose information corresponding to the third image is stored in the target list. The scene recognition model is used to process the image corresponding to each pose information in the target list to obtain the environmental information and feature vector of the robot's environment; The target database is constructed based on each pose information in the target list, the environmental information of the robot's environment, and the feature vector.

8. The method according to any one of claims 2 to 6, wherein, The robot relocation method further includes: Acquire sample images, which are images acquired by the image acquisition device under different operating conditions; Each of the sample images is classified to determine the room type and floor material corresponding to each sample image, thus obtaining training samples; The pre-defined network model is used to train the network, resulting in the trained network model. The trained network model is transformed to obtain the scene recognition model.

9. A repositioning device for a robot, characterized in that The robot includes an image acquisition device and a lidar, and the robot's repositioning device includes: The acquisition unit is configured to acquire a first image output by the image acquisition device upon receiving a relocation request, wherein the first image includes an image of the environment in which the robot is located; A determining unit is configured to determine the target pose information of the robot based at least in part on the first image; The positioning unit is used to control the lidar to perform repositioning based on the target pose information.

10. A repositioning device for a robot, characterized in that It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the robot relocation method as described in any one of claims 1 to 8.

11. A readable storage medium, characterized by, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the robot relocation method as described in any one of claims 1 to 8.

12. A robot, characterized in that include: The robot repositioning device as described in claim 9 or 10; and / or The readable storage medium as described in claim 11.

13. The robot of claim 12, wherein, The robots include robotic vacuum cleaners.

Images