Control method and device of mobile robot, mobile robot

Abstract

The application discloses a kind of control method and device of mobile robot, mobile robot.Therein, the method includes: by the information acquisition equipment of mobile robot carried, current point cloud data in the predetermined range of mobile robot is collected;When target object disappears according to current point cloud data, the information corresponding to target object is deleted from the first map, to obtain the second map, wherein, target object is the object that needs to avoid on initial travel path of mobile robot, initial travel path is generated based on the first map, the first map is generated according to historical data, historical data is the data collected in last data acquisition cycle;According to the second map, initial travel path is updated, to obtain the latest travel path;Control mobile robot and carry object to target position according to the latest travel path.The application solves the technical problem that the information retained in the map of the mobile robot driving dependence in the related art will not be updated in time, and redundant planning of navigation path is easy.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and more specifically, to a control method and apparatus for a mobile robot, and a mobile robot. Background Technology

[0002] Currently, in the control of mobile robots, it is impossible to remove the information of moving objects they record. As a result, when the robot travels to the location of the recorded moving object again, it will still pass by detouring, which leads to longer travel path planning and travel time, and thus increases time and energy costs.

[0003] There is currently no effective solution to the problem that the information stored in the map on which mobile robots rely for navigation is not updated in a timely manner, which can easily lead to redundant planning of navigation paths. Summary of the Invention

[0004] This invention provides a control method and apparatus for a mobile robot, as well as a mobile robot, to at least solve the technical problem in related technologies that the information retained by the map on which the mobile robot relies for driving is not updated in a timely manner, which easily leads to redundant planning of the navigation path.

[0005] According to one aspect of the present invention, a control method for a mobile robot is provided, comprising: collecting current point cloud data within a predetermined range of the mobile robot using an information acquisition device mounted on the mobile robot; deleting information corresponding to the target object from a first map when it is determined from the current point cloud data to obtain a second map, wherein the target object is an object that the mobile robot needs to avoid on its initial driving path, the initial driving path being generated based on the first map, the first map being generated based on historical data, the historical data being data collected in the previous data acquisition cycle; updating the initial driving path according to the second map to obtain a latest driving path; and controlling the mobile robot to transport the object to the target location according to the latest driving path.

[0006] Optionally, the information collection device mounted on the mobile robot collects current point cloud data within a predetermined range of the mobile robot, including: triggering the information collection device to collect current point cloud data within a predetermined range of the mobile robot at a predetermined frequency during the mobile robot's travel along the initial travel path.

[0007] Optionally, determining the disappearance of the target object based on the current point cloud data includes: inputting the current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and a detection value corresponding to the point cloud data, the point cloud data being the input of the information detection model, and the detection value being the output of the information detection model; obtaining the output result of the information detection model; and determining that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object.

[0008] Optionally, before inputting the current point cloud data into the information detection model, the control method of the mobile robot further includes: generating multiple sets of point cloud data from the training data based on the feature information of the target object, wherein the feature information includes at least: the type of the target object and the size of the target object; generating a first detection value representing the target object disappearing from the map, and generating a second detection value representing the target object not disappearing from the map; determining the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and training the multiple sets of training data to obtain the information detection model.

[0009] Optionally, when it is determined that the target object has disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain the second map, including: deleting the point cloud data corresponding to the target object from the current point cloud data to obtain target point cloud data; projecting the target point cloud data into a distance image, and separating the planar point cloud data and non-planar point cloud data from the distance image; extracting corner and face features from the non-planar point cloud data to obtain feature data of the non-planar point cloud data; and performing feature matching on the feature data of point cloud data in multiple consecutive frames to obtain the second map.

[0010] Optionally, feature matching is performed on the feature data of point cloud data in multiple consecutive frames to obtain the second map, including: obtaining the travel prediction information of the mobile robot's odometer and the pose prediction information of the second map; and correcting the third map obtained by feature matching according to the travel prediction information and the pose prediction information to obtain the second map.

[0011] Optionally, controlling the mobile robot to transport the object to the target location according to the latest travel path includes: when it is determined that the mobile robot has moved to the object according to the latest travel path, controlling an image acquisition device to acquire an image of the object; determining the object's feature information based on the object image, wherein the object's feature information includes at least: the object's size, the object's position information, and the object's placement information; determining the relative position information between the mobile robot and the object based on the feature information; and after controlling the mobile robot to load the object according to the relative position information, controlling the mobile robot to transport the object to the target location according to the latest travel path.

[0012] Optionally, controlling the mobile robot to load the object according to the relative position information includes: after controlling the mobile robot to travel under the object according to the relative position information, calibrating the object and the handling component mounted on the mobile robot according to the relative position information; and using the calibrated handling component to load the object into the loading area of ​​the mobile robot.

[0013] According to another aspect of the present invention, a control device for a mobile robot is also provided, comprising: a data acquisition unit, configured to acquire current point cloud data within a predetermined range of the mobile robot via an information acquisition device mounted on the mobile robot; a processing unit, configured to delete information corresponding to the target object from a first map and obtain a second map when it is determined from the current point cloud data that a target object has disappeared, wherein the target object is an object that the mobile robot needs to avoid on its initial driving path, the initial driving path being generated based on the first map, the first map being generated based on historical data, the historical data being data acquired in the previous data acquisition cycle; an update unit, configured to update the initial driving path according to the second map to obtain a latest driving path; and a control unit, configured to control the mobile robot to transport the object to the target location according to the latest driving path.

[0014] Optionally, the acquisition unit includes: a triggering module, configured to trigger the information acquisition device to acquire current point cloud data within a predetermined range of the mobile robot at a predetermined frequency during the process of the mobile robot traveling along the initial travel path.

[0015] Optionally, the processing unit includes: a processing module, configured to input the current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and a detection value corresponding to the point cloud data, the point cloud data being the input of the information detection model, and the detection value being the output of the information detection model; an acquisition module, configured to acquire the output result of the information detection model; and a first determination module, configured to determine that the target object has disappeared when the output result is consistent with the detection value corresponding to when the target object disappears.

[0016] Optionally, the control device of the mobile robot further includes: a first generation module, configured to generate multiple sets of point cloud data from the training data based on the feature information of the target object before inputting the current point cloud data into the information detection model, wherein the feature information includes at least: the type of the target object and the size of the target object; a second generation module, configured to generate a first detection value representing the target object disappearing from the map, and generate a second detection value representing the target object not disappearing from the map; a second determination module, configured to determine the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and an acquisition module, configured to train the multiple sets of training data to obtain the information detection model.

[0017] Optionally, the processing unit includes: a deletion module, configured to delete the point cloud data corresponding to the target object from the current point cloud data to obtain target point cloud data; a separation module, configured to project the target point cloud data into a distance image and separate planar point cloud data and non-planar point cloud data from the distance image; an extraction module, configured to extract corner and face features from the non-planar point cloud data to obtain feature data of the non-planar point cloud data; and a matching module, configured to perform feature matching on the feature data of point cloud data in multiple consecutive frames to obtain the second map.

[0018] Optionally, the matching module includes: a first acquisition submodule, used to acquire the travel prediction information of the mobile robot's odometer and the pose prediction information of the second map; and a second acquisition submodule, used to correct the third map obtained by feature matching based on the travel prediction information and the pose prediction information to obtain the second map.

[0019] Optionally, the control unit includes: a control module, configured to control an image acquisition device to acquire an image of the object when it is determined that the mobile robot has moved to the object according to the latest travel path; a third determining module, configured to determine the feature information of the object based on the object image, wherein the feature information of the object includes at least: the size of the object, the position information of the object, and the placement information of the object; a fourth determining module, configured to determine the relative position information between the mobile robot and the object based on the feature information; and a control module, configured to control the mobile robot to load the object according to the relative position information and then control the mobile robot to transport the object to the target location according to the latest travel path.

[0020] Optionally, the control module includes: a control submodule, configured to calibrate the object and the transport component mounted on the mobile robot according to the relative position information after controlling the mobile robot to travel under the object according to the relative position information; and a loading submodule, configured to load the object into the cargo area of ​​the mobile robot using the calibrated transport component.

[0021] According to another aspect of the present invention, a mobile robot is also provided, which uses the control method for the mobile robot described in any one of the above embodiments.

[0022] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein the program executes the control method for a mobile robot described in any of the above embodiments.

[0023] According to another aspect of the present invention, a processor is also provided, the processor being configured to run a program, wherein the program, when running, executes the control method for the mobile robot described in any of the above embodiments.

[0024] In this embodiment of the invention, the mobile robot collects current point cloud data within a predetermined range using an information acquisition device mounted on it. When the target object is determined to have disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map, resulting in a second map. The target object is an object that the mobile robot needs to avoid on its initial travel path. The initial travel path is generated based on the first map, which is generated from historical data collected in the previous data acquisition cycle. The initial travel path is updated based on the second map to obtain the latest travel path. The mobile robot is then controlled to transport the object to the target location according to the latest travel path. This invention provides a mobile robot control method that achieves real-time updates to the map used for path planning, thereby deleting information corresponding to objects that no longer need to be avoided or yielded on the mobile robot's travel path. This reduces redundant path planning by the mobile robot, improving its work efficiency and solving the technical problem in related technologies where the information retained by the map on which the mobile robot relies is not updated in a timely manner, easily leading to redundant planning of the navigation path. Attached Figure Description

[0025] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0026] Figure 1 This is a hardware structure block diagram of a mobile terminal for a mobile robot control method according to an embodiment of the present invention.

[0027] Figure 2 This is a flowchart of a control method for a mobile robot according to an embodiment of the present invention;

[0028] Figure 3 This is a flowchart of an optional control method for a mobile robot according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of a control device for a mobile robot according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] As described in the background section, in related technologies, mobile robots cannot delete information about moving objects. When the robot returns to the same location, it may detour, leading to excessively long travel times and increased time and energy costs. The embodiments of this invention provide a control method and apparatus for a mobile robot, a mobile robot, a computer-readable storage medium, and a processor.

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0034] The methods and embodiments provided in this invention can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a mobile robot control method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the mobile robot control method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] According to an embodiment of the present invention, a method embodiment for controlling a mobile robot is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] Figure 2 This is a flowchart of a control method for a mobile robot according to an embodiment of the present invention, such as... Figure 2 As shown, the control method for this mobile robot includes the following steps:

[0038] Step S202: Collect current point cloud data within a predetermined range of the mobile robot using the information collection device mounted on the mobile robot.

[0039] Optionally, the mobile robot here can be an automated guided vehicle (AGV) used in a factory to move goods. Of course, it can also be a robot with other functions.

[0040] Optionally, the information acquisition device here can be a lidar, such as a multi-line lidar. Other types of information acquisition devices are also possible, and no specific limitation is made here.

[0041] In one alternative embodiment, when the mobile robot is required to perform a cargo handling task, an instruction can be sent to the mobile robot to trigger the multi-line lidar on the mobile robot to collect point cloud data within its predetermined range.

[0042] Step S204: When it is determined that the target object has disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain the second map. The target object is the object that the mobile robot needs to avoid on the initial driving path. The initial driving path is generated based on the first map. The first map is generated based on historical data, which is the data collected in the previous data collection cycle.

[0043] Optionally, the target object mentioned above can be a person ahead of the mobile robot on its travel path or the goods it needs to move. For example, when the mobile robot last performed a moving task, there were still goods at location A; it's possible that another mobile robot has already moved the goods at location A in the next moment. The goods at location A are objects that the mobile robot needs to avoid according to its initial travel path. In this case, the location information corresponding to the goods at location A needs to be deleted from the map used to generate the initial travel path. This way, the mobile robot no longer needs to avoid location A during its journey, reducing the detour distance and improving its work efficiency.

[0044] Furthermore, in this embodiment of the invention, after obtaining the second map, the second map can be uploaded to a cloud server for backup, so as to facilitate subsequent data traceability.

[0045] In addition, after obtaining the second map, it can be synchronized with other mobile robots to facilitate subsequent collaborative work among multiple robots.

[0046] Step S206: Update the initial driving path according to the second map to obtain the latest driving path.

[0047] In this embodiment, the initial travel path can be updated based on the updated map, thereby reducing detours in the latest travel path. The mobile robot can reach the location of the goods to be transported with the highest efficiency by following the latest travel path.

[0048] Step S208: Control the mobile robot to transport the object to the target location according to the latest travel path.

[0049] In this embodiment, the mobile robot can be controlled to transport the object to the target location according to the latest travel path.

[0050] As can be seen from the above, in this embodiment of the invention, the current point cloud data within a predetermined range of the mobile robot can be collected by the information collection device mounted on the mobile robot; when it is determined that the target object has disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain the second map. The target object is the object that needs to be avoided on the initial driving path of the mobile robot. The initial driving path is generated based on the first map, which is generated based on historical data. The historical data is the data collected in the previous data collection cycle; the initial driving path is updated according to the second map to obtain the latest driving path; the mobile robot is controlled to transport the object to the target position according to the latest driving path. This realizes the real-time updating of the map on which the mobile robot's path planning is based, thereby deleting the information corresponding to the object that no longer needs to be avoided or yielded on the mobile robot's driving path, reducing the redundant path planning generated by the mobile robot, and achieving the technical effect of improving the working efficiency of the mobile robot.

[0051] Therefore, the technical solution provided by the embodiments of the present invention solves the technical problem in the related art that the information retained by the map on which the mobile robot relies for driving is not updated in a timely manner, which easily leads to redundant planning of the navigation path.

[0052] According to the above embodiments of the present invention, collecting current point cloud data within a predetermined range of the mobile robot by means of an information collection device mounted on the mobile robot may include: triggering the information collection device to collect current point cloud data within a predetermined range of the mobile robot at a predetermined frequency during the process of the mobile robot traveling along the initial travel path.

[0053] In this embodiment, while the mobile robot is traveling along its current path, a multi-line lidar is triggered to collect point cloud data within a predetermined range of the mobile robot at a predetermined frequency; that is, the current point cloud data. This predetermined frequency can be determined based on the mobile robot's speed, thereby improving the efficiency of point cloud data acquisition and enabling better acquisition of the data.

[0054] According to the above embodiments of the present invention, determining the disappearance of a target object based on current point cloud data includes: inputting the current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and a detection value corresponding to the point cloud data, the point cloud data being the input of the information detection model and the detection value being the output of the information detection model; obtaining the output result of the information detection model; and determining that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object.

[0055] In this embodiment, a dataset of relevant moving objects, such as people or goods, can be created by multi-line LiDAR. The collected dataset is then added to the target detection algorithm for deep learning training, and the effectiveness of the constructed dataset is verified. Finally, the trained deep learning model, i.e., the information detection model, is derived.

[0056] In this embodiment of the invention, point cloud data serves as the input to the information detection model, and the detection value corresponding to the point cloud data is the output of the information detection model. That is, the point cloud data is input into a basic network model, and the basic network model is trained to obtain the aforementioned information detection model. Using this information detection model, it is possible to quickly determine whether an object or person has disappeared from the path of a mobile robot, further improving the working efficiency of the mobile robot.

[0057] In the above embodiments of the present invention, before inputting the current point cloud data into the information detection model, the control method of the mobile robot may further include: generating point cloud data from multiple sets of training data based on the feature information of the target object, wherein the feature information includes at least: the type of the target object and the size of the target object; generating a first detection value representing the point cloud data when the target object disappears from the map, and generating a second detection value representing the point cloud data when the target object does not disappear from the map; determining the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and training the multiple sets of training data to obtain the information detection model.

[0058] In this embodiment, sample point cloud data for training an information detection model can be generated based on objects that may appear on the mobile robot's path. Simultaneously, sample detection values ​​corresponding to when the sample point cloud data disappears in the map are generated. The sample point cloud data and sample detection values ​​are used as multiple sets of training data. The basic network model is trained using these multiple sets of training data to obtain the information detection model.

[0059] It should be noted that the basic network model here can be a convolutional neural network model, a recurrent neural network model, etc. No specific limitations are specified here.

[0060] According to the above embodiments of the present invention, when it is determined that the target object has disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain the second map, including: deleting the point cloud data corresponding to the target object from the current point cloud data to obtain target point cloud data; projecting the target point cloud data into a distance image, and separating the planar point cloud data and non-planar point cloud data from the distance image; extracting corner and surface features from the non-planar point cloud data to obtain feature data of the non-planar point cloud data; and performing feature matching on the feature data of point cloud data in multiple consecutive frames to obtain the second map.

[0061] In this embodiment, the point cloud data corresponding to objects or people that have disappeared can be deleted from the current point cloud data collected by the multi-line LiDAR on the mobile robot to obtain the target point cloud data. Then, the target point cloud data is projected into a distance map, and the ground points and non-ground points in the target point cloud data are separated from the distance map. The remaining non-ground point cloud data is then clustered. Next, corner and face features are extracted from the segmented point cloud data, and feature matching is performed between corner and face points between consecutive frames to find the pose transformation matrix between consecutive frames. The features are further processed, and then registered in the global point cloud map to obtain the second map mentioned above, that is, the updated map.

[0062] According to the above embodiments of the present invention, feature matching is performed on the feature data of point cloud data of multiple consecutive frames to obtain a second map, including: obtaining travel prediction information of the mobile robot's odometer and pose prediction information of the second map; correcting the third map obtained by feature matching based on the travel prediction information and pose prediction information to obtain the second map.

[0063] In this embodiment, the final pose estimate can be output through the odometry estimation performed by the multi-line radar and the pose estimation of the constructed map, and finally the constructed map can be output.

[0064] Specifically, the point cloud data from two consecutive frames detected by multi-line radar can be optimized, as these two frames overlap. Since the radar pose is fixed from the first frame, subsequent keyframes are estimated based on this, with an overall error estimation performed every fixed number of frames. As shown in the figure, there is a significant overlap between adjacent keyframes. Odometry estimates are obtained by estimating the mileage based on the poses of adjacent different point clouds.

[0065] Furthermore, the pose estimation described above is based on a map stitched together from the poses of several adjacent keyframes to obtain the pose of a local map. After all keyframes have undergone a loop closure, a comprehensive map optimization is performed to complete the final pose estimation and ultimately build the map. A keyframe is a frame of data that satisfies a sufficient number of features.

[0066] According to the above embodiments of the present invention, controlling a mobile robot to transport an object to a target location according to the latest travel path includes: when it is determined that the mobile robot has moved to the object according to the latest travel path, controlling an image acquisition device to acquire an image of the object; determining the object's feature information based on the object image, wherein the object's feature information includes at least: the object's size, the object's position information, and the object's placement information; determining the relative position information between the mobile robot and the object based on the feature information; and after controlling the mobile robot to load the object according to the relative position information, controlling the mobile robot to transport the object to the target location according to the latest travel path.

[0067] In this embodiment, when the mobile robot moves to the object according to the latest travel path, the image acquisition device can be controlled to acquire an image of the object. After determining the object's feature information based on the object image, the relative position information between the mobile robot and the object can be determined based on the feature information. Thus, the mobile robot can be controlled to load the object according to the relative position information, and then the mobile robot can be controlled to transport the object to the target location according to the latest travel path to complete the cargo transportation task.

[0068] According to the above embodiments of the present invention, controlling a mobile robot to load an object based on relative position information includes: after controlling the mobile robot to travel under the object based on the relative position information, calibrating the object and the transport component mounted on the mobile robot based on the relative position information; and using the calibrated transport component to load the object into the loading area of ​​the mobile robot.

[0069] In this embodiment, an AGV (Automated Guided Vehicle) is used as an example to illustrate the process. When the AGV arrives near an object, the depth camera starts working and is adjusted to acquire object features. It observes and evaluates to determine the object's size, position, and placement information, calculates the spatial coordinates (i.e., relative position information) between the AGV and the object, and moves the AGV to the bottom of the object. The object and the AGV's transport frame are then aligned, the transport frame is raised, the object is lifted by the AGV, and the object is transported to the designated location. This allows for better loading of large and heavy container objects, effectively overcoming the drawbacks of using grippers for objects larger than large and heavy containers, which wastes manpower and poses safety hazards.

[0070] The control method for a mobile robot provided in the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this embodiment, the mobile robot is an AGV (Automated Guided Vehicle) as an example. Figure 3 This is a flowchart of an optional mobile robot control method according to an embodiment of the present invention, such as... Figure 3 As shown, during the AGV's movement, a target detection algorithm detects moving objects. The relevant point cloud data of these moving objects is then removed according to the detected regions. A multi-line LiDAR Lego_LOAM algorithm is used for semantic segmentation to project the point cloud into a distance image, separating ground points from non-ground points. Simultaneously, the remaining point cloud is clustered. Edge and face features are extracted from the segmented point cloud, and feature matching is performed between edge and face points in consecutive frames to find the pose transformation matrix between consecutive frames. The features are further processed, and then registered in the global point cloud map. Finally, a map is generated. After map creation, path planning is performed. The robot arrives according to the received target location information. The multi-line LiDAR matches the current frame with the surrounding environment to achieve real-time robot localization and understand the AGV's pose.

[0071] When the AGV reaches the vicinity of the object, the depth camera starts working and is adjusted to acquire the object's features. It observes and evaluates to determine the object's spatial size, position, and placement information, calculates the spatial coordinates between the AGV and the object, moves the AGV to the bottom of the object, calibrates the object and the AGV's transport frame, raises the transport frame, lifts the object, moves the AGV, and transports it to the designated position.

[0072] The mobile robot control method provided in the above embodiments of the present invention removes information about dynamic objects and then uses an optimized algorithm to build a map, thereby completing the path planning of the mobile robot. Furthermore, it acquires the pose of the container using an image acquisition device, calculates the spatial coordinates and rotation information between the mobile robot and the container, completes the robot's rotation, reaches the designated position, and transports the object. This method effectively solves the following problems: 1) The information retained in the map of moving objects creates redundant planning for the navigation path; 2) Using a gripper is unsuitable for objects larger than large, heavy containers. On the one hand, by removing information about moving objects, the mapping of the vehicle is not affected, and detours are avoided when the robot returns to the same location, optimizing the travel distance and time. On the other hand, it makes lifting large, heavy objects more convenient and reduces safety hazards.

[0073] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0075] According to an embodiment of the present invention, a control device for a mobile robot for implementing the above-described control method for a mobile robot is also provided. Figure 4 This is a schematic diagram of a control device for a mobile robot according to an embodiment of the present invention, such as... Figure 4As shown, the device includes: a data acquisition unit 41, a processing unit 43, an update unit 45, and a control unit 47. The control device of the mobile robot will be described below.

[0076] The acquisition unit 41 is used to acquire current point cloud data within a predetermined range of the mobile robot through the information acquisition device mounted on the mobile robot.

[0077] The processing unit 43 is used to delete the information corresponding to the target object from the first map when it is determined that the target object has disappeared based on the current point cloud data, and obtain the second map. The target object is the object that needs to be avoided on the initial driving path of the mobile robot. The initial driving path is generated based on the first map, which is generated based on historical data. The historical data is the data collected in the previous data collection cycle.

[0078] Update unit 45 is used to update the initial driving path based on the second map to obtain the latest driving path.

[0079] Control unit 47 is used to control the mobile robot to transport the object to the target location according to the latest travel path.

[0080] It should be noted that the above-mentioned acquisition unit 41, processing unit 43, update unit 45 and control unit 47 correspond to steps S202 to S208 in the above embodiments. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments.

[0081] As can be seen from the above, in the solution described in the above embodiments of the present invention, the acquisition unit can collect the current point cloud data within a predetermined range of the mobile robot through the information acquisition device mounted on the mobile robot; then, when the processing unit determines that the target object has disappeared based on the current point cloud data, it deletes the information corresponding to the target object from the first map to obtain the second map. The target object is the object that needs to be avoided on the initial driving path of the mobile robot. The initial driving path is generated based on the first map, which is generated based on historical data. The historical data is the data collected in the previous data acquisition cycle; then, the update unit updates the initial driving path based on the second map to obtain the latest driving path; finally, the control unit controls the mobile robot to transport the object to the target position according to the latest driving path. This achieves real-time updating of the map on which the mobile robot's path planning is based, thereby deleting the information corresponding to objects that no longer need to be avoided or yielded on the mobile robot's driving path, reducing redundant path planning generated by the mobile robot, and achieving the technical effect of improving the working efficiency of the mobile robot.

[0082] Therefore, the technical solution provided by the embodiments of the present invention solves the technical problem in the related art that the information retained by the map on which the mobile robot relies for driving is not updated in a timely manner, which easily leads to redundant planning of the navigation path.

[0083] In one optional embodiment, the acquisition unit includes: a triggering module, configured to trigger an information acquisition device to acquire current point cloud data of the mobile robot within a predetermined range at a predetermined frequency during the process of the mobile robot traveling along the initial travel path.

[0084] In one optional embodiment, the processing unit includes: a processing module, configured to input current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, each set of training data including: point cloud data and corresponding detection values, the point cloud data being the input of the information detection model, and the detection values ​​being the output of the information detection model; an acquisition module, configured to acquire the output result of the information detection model; and a first determination module, configured to determine that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object.

[0085] In an optional embodiment, the control device of the mobile robot further includes: a first generation module, configured to generate point cloud data from multiple sets of training data based on the feature information of the target object before inputting the current point cloud data into the information detection model, wherein the feature information includes at least: the type of the target object and the size of the target object; a second generation module, configured to generate a first detection value representing the target object disappearing from the map, and generate a second detection value representing the target object not disappearing from the map; a second determination module, configured to determine the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and an acquisition module, configured to train the multiple sets of training data to obtain the information detection model.

[0086] In one optional embodiment, the processing unit includes: a deletion module, configured to delete the point cloud data corresponding to the target object from the current point cloud data to obtain target point cloud data; a separation module, configured to project the target point cloud data into a distance image and separate planar point cloud data and non-planar point cloud data from the distance image; an extraction module, configured to extract corner and face features from the non-planar point cloud data to obtain feature data of the non-planar point cloud data; and a matching module, configured to perform feature matching on the feature data of point cloud data in multiple consecutive frames to obtain a second map.

[0087] In one optional embodiment, the matching module includes: a first acquisition submodule, used to acquire travel prediction information from the mobile robot's odometer and pose prediction information from the second map; and a second acquisition submodule, used to correct the third map obtained by feature matching based on the travel prediction information and pose prediction information to obtain the second map.

[0088] In one optional embodiment, the control unit includes: a control module, configured to control an image acquisition device to acquire an image of the object when it is determined that the mobile robot has moved to the object according to the latest travel path; a third determination module, configured to determine the object's feature information based on the object image, wherein the object's feature information includes at least: the object's size, the object's position information, and the object's placement information; a fourth determination module, configured to determine the relative position information between the mobile robot and the object based on the feature information; and a control module, configured to control the mobile robot to load the object according to the relative position information and then control the mobile robot to transport the object to the target location according to the latest travel path.

[0089] In one optional embodiment, the control module includes: a control submodule, configured to calibrate the object and the transport component mounted on the mobile robot according to the relative position information after controlling the mobile robot to travel under the object; and a loading submodule, configured to load the object into the loading area of ​​the mobile robot using the calibrated transport component.

[0090] According to another aspect of the present invention, a mobile robot is also provided, which uses the mobile robot control method of any of the above embodiments.

[0091] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein the program executes the control method of the mobile robot described in any of the above embodiments.

[0092] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any communication device in a group of communication devices.

[0093] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: collecting current point cloud data within a predetermined range of the mobile robot using an information acquisition device mounted on the mobile robot; when it is determined from the current point cloud data that the target object has disappeared, deleting the information corresponding to the target object from the first map to obtain a second map, wherein the target object is an object that needs to be avoided on the initial driving path of the mobile robot, the initial driving path is generated based on the first map, the first map is generated based on historical data, and the historical data is data collected in the previous data acquisition cycle; updating the initial driving path according to the second map to obtain the latest driving path; and controlling the mobile robot to transport the object to the target location according to the latest driving path.

[0094] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: during the process of the mobile robot traveling along the initial travel path, triggering the information acquisition device to collect the current point cloud data within a predetermined range of the mobile robot at a predetermined frequency.

[0095] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: inputting current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and corresponding detection values, where the point cloud data is the input of the information detection model and the detection values ​​are the output of the information detection model; obtaining the output result of the information detection model; and determining that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object.

[0096] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: generating point cloud data from multiple sets of training data based on the feature information of the target object, wherein the feature information includes at least: the type of the target object and the size of the target object; generating a first detection value representing the point cloud data when the target object disappears from the map, and generating a second detection value representing the point cloud data when the target object does not disappear from the map; determining the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and training the multiple sets of training data to obtain an information detection model.

[0097] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: deleting the point cloud data corresponding to the target object from the current point cloud data to obtain target point cloud data; projecting the target point cloud data into a distance image, and separating planar point cloud data and non-planar point cloud data from the distance image; extracting corner and face features from the non-planar point cloud data to obtain feature data of the non-planar point cloud data; and performing feature matching on the feature data of point cloud data in multiple consecutive frames to obtain a second map.

[0098] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining travel prediction information from the mobile robot's odometer and pose prediction information from the second map; correcting the third map obtained by feature matching based on the travel prediction information and pose prediction information to obtain the second map.

[0099] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: when it is determined that the mobile robot has moved to the object according to the latest travel path, the image acquisition device is controlled to acquire an image of the object; the object's feature information is determined based on the object image, wherein the object's feature information includes at least: the object's size, the object's position information, and the object's placement information; the relative position information between the mobile robot and the object is determined based on the feature information; after controlling the mobile robot to load the object according to the relative position information, the mobile robot is controlled to transport the object to the target location according to the latest travel path.

[0100] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: after controlling the mobile robot to travel under the object according to the relative position information, calibrating the object and the handling component carried by the mobile robot according to the relative position information; and using the calibrated handling component to load the object into the loading area of ​​the mobile robot.

[0101] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes the control method of the mobile robot described above.

[0102] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0103] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0108] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A control method for a mobile robot, characterized in that, include: The mobile robot collects current point cloud data within a predetermined range using information collection equipment mounted on it. When the target object is determined to have disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain the second map. The target object is the object that the mobile robot needs to avoid on its initial driving path. The initial driving path is generated based on the first map, which is generated based on historical data. The historical data is data collected in the previous data collection cycle. The initial driving path is updated based on the second map to obtain the latest driving path; The mobile robot is controlled to transport the object to the target location according to the latest travel path. Determining the disappearance of a target object based on the current point cloud data includes: inputting the current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and a corresponding detection value, where the point cloud data is the input of the information detection model and the detection value is the output of the information detection model; obtaining the output result of the information detection model; and determining that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object. Before inputting the current point cloud data into the information detection model, the method further includes: generating multiple sets of point cloud data from the training data based on the feature information of the target object, wherein the feature information includes at least: the type of the target object and the size of the target object; generating a first detection value representing the target object disappearing from the map using the point cloud data, and generating a second detection value representing the target object not disappearing from the map using the point cloud data; determining the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and training the multiple sets of training data to obtain the information detection model. The method further includes: outputting a final pose estimate based on odometry estimation performed by multi-line radar and pose estimation of the constructed map, and finally outputting the constructed map. This includes optimizing two consecutive frames of point cloud data detected by the multi-line radar, wherein the two consecutive frames of point cloud data overlap. The pose estimation of the constructed map is obtained by stitching together the poses of several adjacent keyframes to obtain the pose of a local map. When all keyframes have a loop closure, an overall map optimization is performed to complete the final pose estimation, thereby completing the map construction. The keyframe is the frame of data that meets a set number of characteristics.

2. The control method for a mobile robot according to claim 1, characterized in that, The mobile robot collects current point cloud data within a predetermined range using an information collection device mounted on it, including: During the process of the mobile robot traveling along the initial travel path, the information collection device is triggered to collect the current point cloud data within a predetermined range of the mobile robot at a predetermined frequency.

3. The control method for a mobile robot according to claim 1, characterized in that, When it is determined that a target object has disappeared based on the current point cloud data, the information corresponding to the target object is deleted from the first map to obtain a second map, including: Delete the point cloud data corresponding to the target object from the current point cloud data to obtain the target point cloud data; The target point cloud data is projected into a distance image, and planar point cloud data and non-planar point cloud data are separated from the distance image. Corner and surface features are extracted from the non-planar point cloud data to obtain the feature data of the non-planar point cloud data. The second map is obtained by performing feature matching on the feature data of point cloud data from multiple consecutive frames.

4. The control method for a mobile robot according to claim 3, characterized in that, The second map is obtained by performing feature matching on feature data from multiple consecutive frames of point cloud data, including: Obtain the trip prediction information from the mobile robot's odometry and the pose prediction information from the second map; The third map obtained by feature matching is corrected based on the trip prediction information and the pose prediction information to obtain the second map.

5. The control method for a mobile robot according to any one of claims 1 to 4, characterized in that, Controlling the mobile robot to transport the object to the target location according to the latest travel path includes: When it is determined that the mobile robot has moved to the object according to the latest travel path, the image acquisition device is controlled to acquire an image of the object; The feature information of the object is determined based on the object image, wherein the feature information of the object includes at least: the size of the object, the position information of the object, and the placement information of the object; The relative position information between the mobile robot and the object is determined based on the feature information; After controlling the mobile robot to load the object according to the relative position information, control the mobile robot to transport the object to the target location according to the latest travel path.

6. The control method for a mobile robot according to claim 5, characterized in that, Controlling the mobile robot to load the object according to the relative position information includes: After controlling the mobile robot to travel under the object according to the relative position information, the object and the handling component carried by the mobile robot are calibrated according to the relative position information; The calibrated handling components are used to load the object into the cargo area of ​​the mobile robot.

7. A control device for a mobile robot, characterized in that, include: The acquisition unit is used to acquire current point cloud data within a predetermined range of the mobile robot through an information acquisition device mounted on the mobile robot. The processing unit is used to delete the information corresponding to the target object from the first map when it is determined that the target object has disappeared based on the current point cloud data, and obtain the second map. The target object is an object that needs to be avoided on the initial driving path of the mobile robot. The initial driving path is generated based on the first map. The first map is generated based on historical data. The historical data is data collected in the previous data collection cycle. An update unit is used to update the initial driving path according to the second map to obtain the latest driving path; The control unit is used to control the mobile robot to transport the object to the target location according to the latest travel path. The processing unit includes: a processing module, configured to input the current point cloud data into an information detection model to process the current point cloud data using the information detection model, wherein the information detection model is trained using multiple sets of training data through machine learning, and each set of training data includes: point cloud data and a detection value corresponding to the point cloud data, wherein the point cloud data is the input of the information detection model, and the detection value is the output of the information detection model; an acquisition module, configured to acquire the output result of the information detection model; and a first determination module, configured to determine that the target object has disappeared when the output result matches the detection value corresponding to the disappearance of the target object. The control device of the mobile robot further includes: a first generation module, configured to, before inputting the current point cloud data into the information detection model, generate multiple sets of point cloud data from the training data based on the feature information of the target object, wherein the feature information includes at least: the type of the target object and the size of the target object; a second generation module, configured to generate a first detection value representing the target object disappearing from the map, and generate a second detection value representing the target object not disappearing from the map; a second determination module, configured to determine the point cloud data, the first detection value, and the second detection value as multiple sets of training data; and an acquisition module, configured to train the multiple sets of training data to obtain the information detection model. The device is also used to output a final pose estimate based on the odometry estimation and pose estimation of the constructed map using multi-line radar, and finally output the constructed map. This includes optimizing two consecutive frames of point cloud data detected by the multi-line radar, wherein the two consecutive frames of point cloud data overlap. The pose estimation of the constructed map is obtained by stitching together the poses of several adjacent keyframes to obtain the pose of a local map. When all keyframes have a complete loop, an overall map optimization is performed to complete the final pose estimation and thus complete the map construction. Here, the keyframe is the frame of data that meets a set number of characteristics.

8. A mobile robot, characterized in that, The mobile robot uses the control method for the mobile robot described in any one of claims 1 to 6.

Images