Mobile robot positioning method and system based on laser and binocular information and medium

Abstract

The embodiment of the invention discloses a mobile robot positioning method and system based on laser and binocular information and a medium, and the method comprises the steps: collecting a binocular camera depth map of a detection environment, and obtaining three-dimensional pseudo laser data according to the binocular camera depth map; and collecting laser radar point cloud data of a detection environment, and performing edge extraction and angle constraint on the laser radar point cloud data to obtain laser radar data. And combining the three-dimensional pseudo laser data and the laser radar data to obtain fused laser data. On the basis of a synchronous positioning and map building algorithm, the position of the mobile robot is positioned by utilizing fused laser data, so that autonomous navigation of the mobile robot in a dynamic environment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of mobile robot positioning, and in particular, to a mobile robot positioning method, system and medium based on laser and binocular information. Background Art

[0002] The mobile robot navigation technology is one of the core technologies in the field of robotics, involving multi-layer architectures such as perception, decision-making, and control, and including main modules such as mapping, positioning, and planning. With the progress of technology and the development of society, mobile robots have been widely applied in industrial and civilian fields, and their autonomous navigation and path planning capabilities have become research hotspots. The SLAM (Simultaneous Localization and Mapping) algorithm is a prerequisite for realizing the autonomous navigation of mobile robots in unknown environments. It involves a robot starting to move from an unknown position in an unknown environment, and during the movement, self-positioning is performed based on position estimation and sensor data, while an incremental map is built.

[0003] In a dynamic environment, a mobile robot needs to accurately perceive the map environment and the positions of other objects to determine its own position. Existing visual SLAM systems such as ORB-SLAM2 are not robust enough in a dynamic environment and are difficult to handle errors caused by dynamic objects, resulting in low accuracy of camera pose estimation and poor mapping accuracy. Summary of the Invention

[0004] Based on this, it is necessary to address the above problems and propose a mobile robot positioning method, system and medium based on laser and binocular information.

[0005] A mobile robot positioning method based on laser and binocular information, the method includes: Collect the depth map of a binocular camera of the detection environment, and obtain three-dimensional pseudo-laser data according to the depth map of the binocular camera; Collect the lidar point cloud data of the detection environment, perform edge extraction and angle constraint on the lidar point cloud data, and obtain lidar data.

[0006] Merge the three-dimensional pseudo-laser data and the lidar data to obtain fused laser data.

[0007] Based on the simultaneous localization and mapping algorithm, perform the positioning of the mobile robot in the detection environment according to the fused laser data.

[0008] Wherein, the collecting the depth map of a binocular camera of the detection environment and obtaining three-dimensional pseudo-laser data according to the depth map of the binocular camera specifically includes: Collect the depth map of a binocular camera of the detection environment.

[0009] For each pixel point in the depth map of the binocular camera, extract the corresponding depth value according to the pixel value of the pixel point.

[0010] Obtain three-dimensional pseudo-lidar data based on the position data and the corresponding depth value of each pixel point in the depth map of the binocular camera, and perform filtering processing on the three-dimensional pseudo-lidar data.

[0011] Among them, the obtaining of the three-dimensional pseudo-lidar data based on the position data and the corresponding depth value of each pixel point in the depth map of the binocular camera specifically includes: Obtain the position data of each pixel point in the depth map of the binocular camera and the position data of the principal point of the binocular camera, where the position data includes the abscissa and the ordinate.

[0012] According to Determine the abscissa of the pixel point in the world coordinate system in the three-dimensional pseudo-lidar data, where X is the abscissa of the pixel point in the world coordinate system, u is the abscissa of the pixel point in the depth map of the binocular camera, is the abscissa of the principal point of the binocular camera, d is the depth value corresponding to each pixel point, and f is the focal length of the binocular camera.

[0013] According to Determine the ordinate of the pixel point in the world coordinate system in the three-dimensional pseudo-lidar data, where is the abscissa of the pixel point in the world coordinate system, is the ordinate of the pixel point in the depth map of the binocular camera, is the ordinate of the principal point of the binocular camera, d is the depth value corresponding to each pixel point, and f is the focal length of the binocular camera.

[0014] According to Determine the depth coordinate of the pixel point in the world coordinate system in the three-dimensional pseudo-lidar data, where is the depth coordinate of the pixel point in the world coordinate system, and d is the depth value corresponding to each pixel point.

[0015] Among them, the acquisition of the lidar point cloud data of the detection environment, the edge extraction and angle constraint of the lidar point cloud data, and the acquisition of the lidar data specifically include: Collect the lidar point cloud data of the detection environment.

[0016] Perform edge extraction on the lidar point cloud data through an edge detection algorithm to obtain point cloud spatial information.

[0017] Constrain the angle of the lidar point cloud data within the viewing angle range of the binocular camera to obtain point cloud viewing angle information.

[0018] Use the point cloud spatial information and the point cloud perspective information as lidar data.

[0019] Among them, the merging of the three-dimensional pseudo-lidar data and the lidar data to obtain the fused lidar data specifically includes: Obtain the rotation matrix and translation vector between the binocular camera coordinate system and the lidar coordinate system.

[0020] Transform the three-dimensional pseudo-lidar data from the binocular camera coordinate system to the lidar coordinate system according to the rotation matrix and the translation vector.

[0021] Merge the transformed three-dimensional pseudo-lidar data and the lidar data to obtain the fused lidar data.

[0022] Among them, based on the simultaneous localization and mapping algorithm, the localization of the mobile robot in the detection environment according to the fused lidar data specifically includes: Publish the fused lidar data as a topic.

[0023] The simultaneous localization and mapping algorithm subscribes to the topic and locates the mobile robot in the detection environment according to the fused lidar data in the topic.

[0024] Among them, before collecting the binocular camera depth map of the detection environment and obtaining the three-dimensional pseudo-lidar data according to the binocular camera depth map, it specifically includes: Obtain the detection environment images at different angles through the binocular camera, and calculate the disparity between the detection environment images at different angles to generate the binocular camera depth map.

[0025] Among them, before collecting the lidar point cloud data of the detection environment, performing edge extraction and angle constraint on the lidar point cloud data to obtain the lidar data, it specifically includes: Obtain the lidar point cloud data of the detection environment through the lidar and preprocess the lidar point cloud data.

[0026] A mobile robot positioning system based on laser and binocular information, the system includes: A three-dimensional pseudo-lidar data acquisition module, configured to collect a binocular camera depth map of a detection environment and obtain three-dimensional pseudo-lidar data according to the binocular camera depth map.

[0027] A lidar data acquisition module, configured to collect lidar point cloud data of a detection environment, perform edge extraction and angle constraint on the lidar point cloud data, and obtain lidar data.

[0028] A fused laser data acquisition module, configured to merge the three-dimensional pseudo-laser data and the lidar data to obtain fused laser data.

[0029] A mobile robot positioning module, configured to perform positioning of the mobile robot in the detected environment based on the simultaneous localization and mapping algorithm according to the fused laser data.

[0030] A computer-readable storage medium stores a computer program, which when executed by a processor causes the processor to execute the steps of the method described above.

[0031] Adopting the embodiments of the present invention has the following beneficial effects: The present invention first obtains three-dimensional pseudo-laser data according to the binocular camera depth map; meanwhile, edge extraction and angle constraint are performed on the lidar point cloud data, and more reliable lidar data is obtained by extracting edge information and angle constraint; subsequently, these pseudo-laser data and the refined lidar data are fused to obtain fused laser data. By fusing the accurate distance measurement of the lidar and the depth information of the binocular camera, the three-dimensional perception ability of the environment is enhanced, low obstacles can be more effectively identified and located, the positioning accuracy of the mobile robot is improved, and at the same time, the adaptability to complex environments and the overall navigation performance are improved by using multi-sensor data. Thus, in a dynamic environment, the fused laser data can reduce the risk of single sensor failure and improve the robustness of the system in complex environments. Finally, based on the simultaneous localization and mapping algorithm, the position of the mobile robot is located using the fused laser data, thereby realizing its autonomous navigation in a dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0033] Among them: Figure 1 is a schematic flowchart of an embodiment of a mobile robot positioning method based on laser and binocular information provided by the present invention; Figure 2 is a schematic flowchart of another embodiment of a mobile robot positioning method based on laser and binocular information provided by the present invention; Figure 3 is a schematic structural diagram of an embodiment of a mobile robot positioning system based on laser and binocular information provided by the present invention; Figure 4Schematic structural diagram of an embodiment of the medium provided by the present invention. Detailed implementation manners

[0034] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0035] As Figure 1 shown, Figure 1 Schematic flowchart of an embodiment of a mobile robot positioning method based on laser and binocular information provided by the present invention. A mobile robot positioning method based on laser and binocular information, the method includes: S101: Collect the depth map of the binocular camera of the detection environment, and obtain three-dimensional pseudo-laser data according to the depth map of the binocular camera.

[0036] Exemplarily, the binocular camera simulates the binocular vision principle of humans, uses two cameras to capture images of the detection environment from different angles, and obtains depth information by calculating the disparity between the two images, and generates a depth map of the binocular camera according to the depth information. The depth map of the binocular camera is a grayscale image, where each pixel value represents the depth of the corresponding point in the scene.

[0037] Further, collect the depth map of the binocular camera of the detection environment. For each pixel point in the depth map of the binocular camera, extract the corresponding depth value according to the pixel value of the pixel point. Obtain three-dimensional pseudo-laser data according to the position data and the corresponding depth value of each pixel point in the depth map of the binocular camera, and perform filtering processing on the three-dimensional pseudo-laser data. The three-dimensional pseudo-laser data includes the abscissa, ordinate, and depth coordinate of the pixel point in the world coordinate system.

[0038] S102: Collect the lidar point cloud data of the detection environment, perform edge extraction and angle constraint on the lidar point cloud data, and obtain lidar data.

[0039] Exemplarily, obtain the lidar point cloud data of the detection environment through the lidar, and perform preprocessing on the lidar point cloud data, such as removing noise, downsampling, Gaussian filtering, etc. Specifically, apply Gaussian filtering to the lidar point cloud data of the detection environment obtained by the lidar to reduce noise. Gaussian filtering is a smoothing filter that smooths the data by convolution with a Gaussian kernel, thereby reducing the high-frequency noise in the data.

[0040] Furthermore, collect the lidar point cloud data of the detection environment, perform edge extraction on the lidar point cloud data through an edge detection algorithm to obtain the point cloud spatial information, and the point cloud spatial information is the obstacle contour in the detection environment. At the same time, determine the viewing range of the binocular camera, then limit the lidar point cloud data within the viewing range of the binocular camera, and exclude the data points outside this range to obtain the point cloud viewing information. Take the point cloud spatial information and the point cloud viewing information as lidar data.

[0041] S103: Merge the three-dimensional pseudo-lidar data and the lidar data to obtain the fused lidar data.

[0042] Exemplarily, perform joint calibration on the binocular camera and the lidar to obtain the rotation matrix and translation vector between the camera coordinate system and the lidar coordinate system. The three-dimensional pseudo-lidar data, the lidar data, as well as the rotation matrix and translation vector between the two coordinate systems are input into the PCL point cloud library, and the PCL point cloud library is used to merge the three-dimensional pseudo-lidar data and the lidar data to obtain the fused lidar data.

[0043] S104: Based on the Simultaneous Localization and Mapping (SLAM) algorithm, localize the mobile robot in the detection environment according to the fused lidar data.

[0044] Exemplarily, publish the fused lidar data as a topic. The SLAM algorithm subscribes to the topic and localizes the mobile robot in the detection environment according to the fused lidar data in the topic. Moreover, the SLAM algorithm constructs the environmental map of the detection environment according to the fused lidar data in the topic. In summary, by publishing this fused lidar data to a specific topic of the SLAM algorithm, the present invention can realize the localization and mapping construction of the mobile robot.

[0045] As can be seen from the above description, the present invention first obtains the three-dimensional pseudo-lidar data according to the depth map of the binocular camera; at the same time, performs edge extraction and angle constraint on the lidar point cloud data, and obtains more reliable lidar data by extracting edge information and angle constraint; subsequently, fuses these pseudo-lidar data and the refined lidar data to obtain the fused lidar data, enhances the three-dimensional perception ability of the environment by fusing the accurate distance measurement of the lidar and the depth information of the binocular camera, can more effectively identify and locate low obstacles, improves the positioning accuracy of the mobile robot, and at the same time utilizes multi-sensor data to enhance the adaptability to complex environments and the overall navigation performance, so that in a dynamic environment, the fused lidar data can reduce the risk of single-sensor failure and improve the robustness of the system in complex environments. Finally, based on the SLAM algorithm, using the fused lidar data, localize the position of the mobile robot, thereby realizing its autonomous navigation in a dynamic environment.

[0046] As Figure 2 shown Figure 2 Figure 2 is a schematic flowchart of another embodiment of a mobile robot positioning method based on laser and binocular information provided by the present invention. A mobile robot positioning method based on laser and binocular information, the method comprising: S201: Obtain detection environment images at different angles through a binocular camera, and calculate the disparity between the detection environment images at different angles to generate a binocular camera depth map.

[0047] Exemplarily, the binocular camera simulates the binocular vision principle of a human, uses two cameras to capture images of the detection environment from different angles, and obtains depth information by calculating the disparity between the two images, and generates a binocular camera depth map according to the depth information.

[0048] S202: Collect the binocular camera depth map of the detection environment.

[0049] S203: For each pixel point in the binocular camera depth map, extract the corresponding depth value according to the pixel value of the pixel point.

[0050] Exemplarily, collect the binocular camera depth map of the detection environment. For each pixel point in the binocular camera depth map, extract the corresponding depth value according to its gray value.

[0051] S204: Obtain three-dimensional pseudo-laser data according to the position data and the corresponding depth value of each pixel point in the binocular camera depth map, and perform filtering processing on the three-dimensional pseudo-laser data.

[0052] Exemplarily, obtain the position data of each pixel point in the binocular camera depth map and the position data of the principal point of the binocular camera. The position data includes the abscissa and the ordinate. The abscissa of the pixel point in the three-dimensional pseudo-laser data in the world coordinate system is determined according to the following formula: ; where X is the abscissa of the pixel point in the world coordinate system, u is the abscissa of the pixel point in the binocular camera depth map, is the abscissa of the principal point of the binocular camera, d is the depth value corresponding to each pixel point, and f is the focal length of the binocular camera.

[0053] The ordinate of the pixel point in the three-dimensional pseudo-laser data in the world coordinate system, where, is the abscissa of the pixel point in the world coordinate system and is determined according to the following formula: ; where, is the ordinate of the pixel point in the binocular camera depth map, is the ordinate of the principal point of the binocular camera, d is the depth value corresponding to each pixel point, and f is the focal length of the binocular camera.

[0054] The depth coordinate of a pixel point in the world coordinate system in the three-dimensional pseudo-lidar data is determined according to the following formula: ; where, is the depth coordinate of the pixel point in the world coordinate system, and d is the depth value corresponding to each pixel point.

[0055] Moreover, due to the complexity of the environment, the three-dimensional pseudo-lidar data obtained from the binocular camera often contains noise and incomplete information. Therefore, it is necessary to perform filtering processing on the three-dimensional pseudo-lidar data to improve the quality and usability of the data. The present invention uses a radius filtering method to perform filtering processing on the three-dimensional pseudo-lidar data. This method determines whether a point is an outlier by checking whether there are enough points in the neighborhood of each point in the three-dimensional pseudo-lidar data. If the number of points in the neighborhood of a point is less than a certain threshold, then this point may not come from the surface of a real object and can be removed. Through the above filtering processing, cleaner and more accurate three-dimensional pseudo-lidar data can be obtained, providing reliable environmental information for the navigation and obstacle avoidance of mobile robots.

[0056] S205: Obtain the lidar point cloud data of the detection environment through a lidar, and perform preprocessing on the lidar point cloud data.

[0057] Exemplarily, obtain the lidar point cloud data of the detection environment through a lidar, and apply Gaussian filtering to the lidar point cloud data generated by the lidar to reduce noise. Gaussian filtering is a smoothing filter that smooths the data by convolving with a Gaussian kernel, thereby reducing the high-frequency noise in the data.

[0058] S206: Collect the lidar point cloud data of the detection environment.

[0059] S207: Perform edge extraction on the lidar point cloud data through an edge detection algorithm to obtain the point cloud spatial information.

[0060] Exemplarily, use the Canny algorithm to identify the edges in the lidar point cloud data and obtain the point cloud spatial information. The Canny algorithm is an edge detection method that identifies edges by calculating the magnitude and direction of the gradient. After performing edge extraction on the lidar data, it helps to identify the obstacle contours in the environment.

[0061] S208: Constrain the angle of the lidar point cloud data within the viewing angle range of the binocular camera to obtain the point cloud viewing angle information.

[0062] Exemplarily, due to the different viewing angles of the lidar and the binocular camera, it is necessary to constrain the lidar data within the measurement range of the binocular camera to ensure the effective fusion of the two sets of data. Specifically, first determine the viewing angle range of the binocular camera, and then limit the lidar point cloud data obtained by the lidar within the viewing angle range of the binocular camera, excluding the data points outside this range. By constraining the angle of the lidar point cloud data, it can be ensured that the lidar data is consistent with the binocular camera data in terms of space and viewing angle, thus providing accurate input data for subsequent data merging and the implementation of the simultaneous localization and mapping algorithm.

[0063] S209: Use the point cloud spatial information and the point cloud viewing angle information as the lidar data.

[0064] S210: Obtain the rotation matrix and the translation vector between the binocular camera coordinate system and the lidar coordinate system.

[0065] Exemplarily, perform joint calibration on the binocular camera and the lidar to obtain the rotation matrix and the translation vector between the camera coordinate system and the lidar coordinate system.

[0066] S211: Transform the three-dimensional pseudo-lidar data from the binocular camera coordinate system to the lidar coordinate system according to the rotation matrix and the translation vector.

[0067] S212: Merge the transformed three-dimensional pseudo-lidar data and the lidar data to obtain the fused lidar data.

[0068] Exemplarily, input the three-dimensional pseudo-lidar data, the lidar data, and the rotation matrix and the translation vector between the two coordinate systems into the PCL point cloud library to fuse the two types of lidar data.

[0069] Specifically, use the functions in the PCL point cloud library for fusion: 1. Use the I / O function of the PCL point cloud library to read or receive two or more point cloud data (the point cloud data includes the three-dimensional pseudo-lidar data and the lidar data).

[0070] 2. Since the point cloud data comes from different coordinate systems, use the pcl::transformPointCloud function to achieve coordinate transformation, and transform the three-dimensional pseudo-lidar data from the binocular camera coordinate system to the lidar coordinate system with the help of the rotation matrix and the translation vector.

[0071] 3. Use the pcl::concatenatePointCloud function to merge the point cloud data after position transformation to obtain the fused lidar data.

[0072] S213: Publish the fused lidar data as a topic.

[0073] S214: Subscribe to the topic of the Simultaneous Localization and Mapping (SLAM) algorithm, and detect the location of the mobile robot in the environment based on the fused lidar data in the topic.

[0074] It should be noted that steps S213 - S214 have been described in detail in the Figure 1 illustrated implementation scenario and will not be elaborated here.

[0075] As can be seen from the above description, the present invention first uses a binocular camera to obtain detection environment images from different angles, generates a binocular camera depth map based on the disparity of the detection environment images from different angles, obtains three-dimensional pseudo-lidar data according to the position data and corresponding depth values of each pixel point in the binocular camera depth map, and removes noise and incomplete information through filtering to obtain more reliable three-dimensional pseudo-lidar data. Then, Gaussian filtering is used to reduce the noise of the lidar point cloud data, and an edge detection algorithm is used to extract the environmental boundary to obtain the point cloud spatial information. Subsequently, the angle of the lidar point cloud data is constrained within the field of view of the binocular camera to ensure the data consistency between the two, and the point cloud perspective information is obtained. The point cloud spatial information and the point cloud perspective information are used as lidar data. Finally, with the help of the PCL point cloud library, the processed three-dimensional pseudo-lidar data and the lidar data are fused in the jointly calibrated coordinate system to form more complete and accurate fused lidar data, and the mobile robot's location is achieved by publishing this fused lidar data to a specific topic of the Simultaneous Localization and Mapping algorithm.

[0076] As Figure 3 shown, Figure 3 is a schematic structural diagram of an embodiment of a mobile robot positioning system based on laser and binocular information provided by the present invention. A mobile robot positioning system 10 based on laser and binocular information, the system includes: A three-dimensional pseudo-lidar data acquisition module 11, configured to collect a binocular camera depth map of the detection environment and obtain three-dimensional pseudo-lidar data according to the binocular camera depth map.

[0077] A lidar data acquisition module 12, configured to collect lidar point cloud data of the detection environment, perform edge extraction and angle constraint on the lidar point cloud data, and obtain lidar data.

[0078] A fused lidar data acquisition module 13, configured to merge the three-dimensional pseudo-lidar data and the lidar data to obtain fused lidar data.

[0079] A mobile robot positioning module 14, configured to detect the location of the mobile robot in the environment based on the Simultaneous Localization and Mapping algorithm according to the fused lidar data.

[0080] Exemplarily, in the three-dimensional pseudo-laser data acquisition module 11, a binocular camera depth map of the detection environment is collected; for each pixel point in the binocular camera depth map, the corresponding depth value is extracted according to the pixel value of the pixel point; three-dimensional pseudo-laser data is obtained according to the position data of each pixel point in the binocular camera depth map and the corresponding depth value, and the three-dimensional pseudo-laser data is filtered. Further, in the lidar data acquisition module 12, lidar point cloud data of the detection environment is collected; edge extraction is performed on the lidar point cloud data through an edge detection algorithm to obtain point cloud spatial information; the angle of the lidar point cloud data is constrained within the viewing angle range of the binocular camera to obtain point cloud viewing angle information; the point cloud spatial information and the point cloud viewing angle information are used as lidar data. Further, in the fused laser data acquisition module 13, the rotation matrix and translation vector between the binocular camera coordinate system and the lidar coordinate system are obtained; the three-dimensional pseudo-laser data is transformed from the binocular camera coordinate system to the lidar coordinate system according to the rotation matrix and translation vector; the transformed three-dimensional pseudo-laser data and the lidar data are merged to obtain fused laser data. Finally, in the mobile robot positioning module 14, the rotation matrix and translation vector between the binocular camera coordinate system and the lidar coordinate system are obtained; the three-dimensional pseudo-laser data is transformed from the binocular camera coordinate system to the lidar coordinate system according to the rotation matrix and translation vector; the transformed three-dimensional pseudo-laser data and the lidar data are merged to obtain fused laser data.

[0081] As Figure 4 shown, Figure 4 is a schematic structural diagram of an embodiment of the medium provided by the present invention. The medium 20 at least includes a memory 21 and a processor 22. At least one computer program is stored in the memory 21, and the computer program is executed by the processor 22 to implement the method as Figure 1 and Figure 2 shown. For the detailed method, reference can be made to the above, and details will not be repeated here. In one embodiment, the medium 20 may be a storage chip, a hard disk, a mobile hard disk, a USB flash drive, an optical disc, or other writable and readable storage tools, or a server, etc.

[0082] In addition, the processes depicted in the drawings do not necessarily have to be in the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0083] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0084] The apparatus, device, non-volatile computer-readable storage medium, and method provided by the embodiments of this specification are corresponding. Therefore, the apparatus, device, and non-volatile computer storage medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device, and non-volatile computer storage medium will not be elaborated here.

[0085] The systems, apparatuses, modules, or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0086] For convenience of description, the above apparatus is described by dividing it into various units according to functions. Of course, when implementing this specification, the functions of each unit may be implemented in one or more software and / or hardware. Those skilled in the art should understand that the embodiments of this specification may be provided as a method, a system, or a computer program product. Therefore, the embodiments of this specification may take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0087] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate an apparatus for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0088] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means that implement the functions specified in one or more of the flow(s) Figure 1 step(s) or block(s) Figure 1 specified in one or more of the block(s) or step(s).

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the flow(s) Figure 1 step(s) or block(s) Figure 1 specified in one or more of the block(s) or step(s).

[0090] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0091] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory such as read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media implemented by any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising", "including" or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0094] This specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0095] Each embodiment in this specification is described in a progressive manner. For parts that are the same or similar among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, they are described relatively simply. For related parts, reference can be made to the description of the method embodiments.

[0096] The above-disclosed are only the preferred embodiments of the present invention. Certainly, the scope of the rights of the present invention cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. A mobile robot positioning method based on laser and binocular information, characterized in that: The method comprises: Collecting a binocular camera depth map of the detection environment, and obtaining three-dimensional pseudo laser data according to the binocular camera depth map; Collecting laser radar point cloud data of the detection environment, performing edge extraction and angle constraint on the laser radar point cloud data, and obtaining laser radar data; Merging the three-dimensional pseudo laser data and the laser radar data to obtain fused laser data; Based on a synchronous positioning and map building algorithm, the mobile robot is positioned in the detection environment according to the fused laser data.

2. The mobile robot positioning method based on laser and binocular information according to claim 1 is characterized in that: The collecting of a binocular camera depth map of the detection environment and obtaining three-dimensional pseudo laser data according to the binocular camera depth map specifically include: Collect the binocular camera depth map of the detection environment; For each pixel point in the binocular camera depth map, extracting a corresponding depth value according to the pixel value of the pixel point; Three-dimensional pseudo laser data is acquired according to the position data of each pixel point in the binocular camera depth map and the corresponding depth value, and the three-dimensional pseudo laser data is filtered.

3. The mobile robot positioning method based on laser and binocular information according to claim 2 is characterized in that: The step of obtaining three-dimensional pseudo laser data according to the position data of each pixel point in the binocular camera depth map and the corresponding depth value specifically includes: Acquire the position data of each pixel in the binocular camera depth map and the position data of the binocular camera principal point, wherein the position data includes a horizontal coordinate and a vertical coordinate; according to Determine the horizontal coordinate of the pixel point in the three-dimensional pseudo laser data in the world coordinate system, wherein X is the horizontal coordinate of the pixel point in the world coordinate system, and u is the horizontal coordinate of the pixel point in the binocular camera depth map, is the horizontal coordinate of the main point of the binocular camera, d is the depth value corresponding to each pixel, and f is the focal length of the binocular camera; according to Determine the ordinate of the pixel point in the three-dimensional pseudo laser data in the world coordinate system, wherein: is the horizontal coordinate of the pixel point in the world coordinate system, is the vertical coordinate of the pixel in the binocular camera depth map, is the ordinate of the main point of the binocular camera, d is the depth value corresponding to each pixel, and f is the focal length of the binocular camera; according to Determine the depth coordinate of the pixel point in the three-dimensional pseudo laser data in the world coordinate system, wherein: is the depth coordinate of the pixel in the world coordinate system, and d is the depth value corresponding to each pixel.

4. The mobile robot positioning method based on laser and binocular information according to claim 1 is characterized in that: The collecting of laser radar point cloud data of the detection environment, performing edge extraction and angle constraint on the laser radar point cloud data, and obtaining the laser radar data specifically includes: Collect LiDAR point cloud data of the detection environment; Extracting edges of the laser radar point cloud data by using an edge detection algorithm to obtain point cloud spatial information; Constraining the angle of the laser radar point cloud data to the viewing angle range of the binocular camera to obtain point cloud viewing angle information; The point cloud spatial information and point cloud viewing angle information are used as laser radar data.

5. The mobile robot positioning method based on laser and binocular information according to claim 2 or 4, characterized in that: The merging of the three-dimensional pseudo laser data and the laser radar data to obtain fused laser data specifically includes: Get the rotation matrix and translation vector between the binocular camera coordinate system and the lidar coordinate system; Transforming the three-dimensional pseudo laser data from the binocular camera coordinate system to the laser radar coordinate system according to the rotation matrix and translation vector; The transformed three-dimensional pseudo laser data and the laser radar data are combined to obtain fused laser data.

6. The mobile robot positioning method based on laser and binocular information according to claim 5 is characterized in that: The method of positioning the mobile robot in the detection environment based on the synchronous positioning and map building algorithm according to the fused laser data specifically includes: publishing the fused laser data as a topic; The synchronous positioning and mapping algorithm subscribes to the topic, and positions the mobile robot in the detection environment according to the fused laser data in the topic.

7. The mobile robot positioning method based on laser and binocular information according to claim 1 is characterized in that: Before acquiring the binocular camera depth map of the detection environment and obtaining the three-dimensional pseudo laser data according to the binocular camera depth map, the method specifically includes: The detection environment images at different angles are acquired by a binocular camera, and the disparity between the detection environment images at different angles is calculated to generate a binocular camera depth map.

8. The mobile robot positioning method based on laser and binocular information according to claim 1 is characterized in that: The step of collecting the laser radar point cloud data of the detection environment and performing edge extraction and angle constraint on the laser radar point cloud data before obtaining the laser radar data specifically includes: The laser radar point cloud data of the detection environment is acquired through the laser radar, and the laser radar point cloud data is preprocessed.

9. A mobile robot positioning system based on laser and binocular information, characterized in that: The system comprises: A three-dimensional pseudo laser data acquisition module is used to collect a binocular camera depth map of the detection environment and acquire three-dimensional pseudo laser data according to the binocular camera depth map; A laser radar data acquisition module is used to collect laser radar point cloud data of the detection environment, perform edge extraction and angle constraint on the laser radar point cloud data, and obtain laser radar data; A fused laser data acquisition module, used for merging the three-dimensional pseudo laser data and the laser radar data to acquire fused laser data; The mobile robot positioning module is used to position the mobile robot in the detection environment according to the fused laser data based on a synchronous positioning and map building algorithm.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

Images