Positioning control method and robot

Abstract

Provided in the present application are a positioning control method and a robot. The positioning control method is applied to a mobile device comprising at least two positioning modules, and the positioning control method comprises: using a first positioning mode to perform positioning; collecting environment information, and performing real-time obstacle analysis on the basis of the environment information; and on the basis of a result of real-time obstacle analysis, determining whether to switch to using a second positioning mode to perform positioning, wherein the first positioning mode is performing positioning on the basis of at least one satellite positioning system, and the second positioning mode is performing positioning on the basis of at least one non-satellite positioning system. In the positioning control method, obstacle detection analysis is performed on the environment information to determine whether positioning data of the first positioning mode is probably to degrade, so as to implement positioning mode switching before the positioning data of the first positioning mode actually degrades.

Description

A positioning control method and robot Technical Field

[0001] This application relates to the field of positioning technology, and in particular to a positioning control method and a robot. Background Technology

[0002] Localization technology is the foundation of automated robot operations. Common localization systems include GNSS localization systems, GNSS-based RTK localization systems, visual localization systems, inertial navigation systems, and wheeled odometry. RTK localization systems have good localization accuracy in relatively open environments, but their localization performance deteriorates when the robot is obstructed by buildings, and pose drift may occur. Visual localization systems have a wide range of applications and are not affected by environmental buildings, but they rely on inter-frame pose transformation for localization, which can lead to error accumulation. Therefore, combining RTK localization and visual localization can compensate for the shortcomings of each.

[0003] In existing technologies, when mobile devices switch between RTK and visual positioning systems, they often determine whether to switch by checking RTK positioning system parameters (whether it is a fixed solution, the number of co-viewing satellites, etc.). Therefore, the positioning mode switch is only performed when the RTK positioning system parameters indicate a deterioration in positioning quality. However, due to the inherent errors of the RTK positioning module and the fact that the mapping between RTK positioning parameters and positioning quality is not absolute, there is sometimes a delay / misjudgment between the actual deterioration of RTK positioning quality and the RTK parameters indicating a deterioration in positioning quality. In addition, in the traditional mode of using satellite positioning systems for positioning, there is also a delay / misjudgment between the actual deterioration of positioning quality and the satellite positioning system indicating a deterioration in positioning quality. During the period from the actual signal interference to the positioning system parameters indicating poor positioning quality and the completion of the positioning mode switch, the robot's positioning data may have drifted, posing a certain risk. Summary of the Invention

[0004] Based on this, this application provides a positioning control method that uses obstacle detection analysis of the mobile device's environmental information to determine whether the mobile device's positioning data may deteriorate, and determines whether to switch the positioning mode based on the judgment result of the positioning data.

[0005] To achieve the above objectives, the embodiments in this specification provide the following technical solutions:

[0006] This application firstly provides a positioning control method as a basic technical solution, applied to mobile devices including at least two positioning systems, the positioning control method comprising:

[0007] Use the first positioning mode for positioning;

[0008] Collect environmental information and perform real-time obstacle analysis based on that information;

[0009] Determine whether to switch to the second positioning mode based on the results of real-time obstacle analysis;

[0010] The first positioning mode is based on at least one satellite positioning system, and the second positioning mode is based on at least one non-satellite positioning system.

[0011] The most basic technical solution described above detects and analyzes the environmental information of mobile devices, and performs real-time obstacle analysis on possible deterioration of satellite positioning signals. This can prevent the mobile device from encountering obstacles due to actual positioning data drift during the period when the satellite positioning signal is actually interfered with, the positioning system parameters indicate poor positioning quality, and the positioning mode is switched.

[0012] Based on the above-mentioned most basic technical solution, the first positioning mode further includes: directly using positioning data from a satellite positioning system for positioning; or fusing positioning data from a satellite positioning system with positioning data from a non-satellite positioning system based on a first parameter to achieve fused data positioning.

[0013] Based on the above-mentioned most basic technical solution, optionally, the real-time obstacle analysis includes: based on the image information in the environmental information, detecting and analyzing obstacles in the environment by analyzing the images, and determining whether to switch to using the second positioning mode for positioning based on the location information of the obstacles.

[0014] In the aforementioned optional positioning control method, further, the step of detecting and analyzing obstacles in the environment by analyzing images includes: acquiring an image dataset of obstacles, labeling the image dataset with obstacles, and constructing and training a neural network model; determining the type and location information of obstacles in the image information based on the trained neural network model; if the location conditions of the obstacles satisfy any one of the first, second, third, and fourth conditions, then switching to positioning using the second positioning mode, wherein the first condition is that at least one obstacle is located above the mobile device; the second condition is that at least one obstacle with a height higher than the mobile device is located within a first distance value range of the mobile device; the third condition is that there are at least two obstacles within the second distance value range of the mobile device; and the fourth condition is that there are at least three non-parallel obstacles within the third distance value range of the mobile device.

[0015] The aforementioned further technical solution, by using a neural network model to detect and analyze obstacles in the environment, can quickly iterate the mobile device's ability to identify the type, quantity, and accuracy of obstacles in order to cope with more complex real-world scenarios. At the same time, it can utilize historical environmental information to build a more complete neural network model. Furthermore, this solution can rely solely on a visual camera or a depth camera, avoiding the use of expensive LiDAR modules.

[0016] Based on the above-mentioned most basic technical solution, optionally, the real-time obstacle analysis includes: based on the point cloud information in the environmental information, detecting and analyzing obstacles in the environment by analyzing the point cloud, and determining whether to switch to using the second positioning mode for positioning based on the position information of the obstacles.

[0017] In the above-mentioned positioning control method based on point cloud information, the step of detecting and analyzing obstacles in the environment by analyzing point cloud information further includes: determining point cloud information whose distance value from the center of the mobile device is less than a first distance value as local environmental information; and analyzing whether there are obstacles whose boundary point cloud is equal to or higher than a first height value based on the local environmental information.

[0018] In the above-mentioned further positioning control method based on point cloud information, optionally, the step of detecting and analyzing obstacles in the environment by analyzing point cloud further includes: based on obstacles whose boundary point cloud is equal to or higher than the first height value, determining whether there are at least two obstacle boundary planes with a plane angle less than the first angle value; if so, switching to positioning using the second positioning mode.

[0019] In the above-mentioned further positioning control method based on point cloud information, optionally, the step of detecting and analyzing obstacles in the environment by analyzing point cloud further includes: performing a two-dimensional projection of the obstacle boundary point cloud based on obstacles whose boundary point cloud is equal to or higher than the first height value; and analyzing whether there are at least two straight lines with an included angle less than the first angle value based on the two-dimensional projection, and if so, switching to positioning using the second positioning mode.

[0020] In the above-mentioned positioning control method based on point cloud information, optionally, the step of detecting and analyzing obstacles in the environment by analyzing point cloud further includes: performing a two-dimensional projection of the obstacle boundary point cloud based on obstacles whose boundary point cloud is equal to or higher than the first height value; based on the two-dimensional projection, analyzing whether there are at least two straight lines with an included angle less than the first angle value; if so, further analyzing whether the minimum distance between the at least two straight lines is less than the second distance value; if so, switching to positioning using the second positioning mode.

[0021] Based on the above-mentioned basic technical solution, the further step of switching to the second positioning mode includes: directly using the positioning data of the non-satellite positioning system to determine the current location.

[0022] Based on the above-mentioned basic technical solution, the further step of switching to the second positioning mode includes: fusing the positioning data of the satellite positioning system and the positioning data of the non-satellite positioning system based on the second parameter, and determining the current position according to the fused data; the second parameter is configured to indicate that the positioning data error of the non-satellite positioning system is less than the positioning data error of the satellite positioning system.

[0023] The aforementioned positioning and control methods based on image information or point cloud information can effectively compensate for the shortcomings of satellite positioning by utilizing laser SLAM or visual camera systems. They also improve the environmental awareness and robustness of positioning information on mobile devices, ensuring the real-time performance and accuracy of positioning mode switching. Among these, the point cloud-based positioning and control method can distinguish the scenarios that most frequently degrade satellite signals by utilizing the geometric structure of environmental point cloud information. This reduces the demands of mobile devices on graphics processing capabilities, enables more efficient switching of positioning modes, reduces waste of hardware resources, and improves the feasibility of the positioning and control method.

[0024] Based on the above-mentioned basic technical solution, the satellite positioning system further includes a GNSS positioning system or a GNSS positioning system with RTK.

[0025] Based on the above-mentioned basic technical solution, the non-satellite positioning system is further described as a visual camera system, a VSLAM system, or a laser SLAM system.

[0026] Based on the above positioning control method, the satellite positioning system and / or the non-satellite positioning system further include an IMU and / or a wheeled odometer to help improve positioning accuracy.

[0027] This application provides a robot in a second aspect. The robot is a mobile device and includes a satellite positioning system, a non-satellite positioning system, an analysis module, and a positioning mode switching module. The robot performs positioning based on the satellite positioning system or based on a combination of the satellite positioning system and the non-satellite positioning system in a first positioning mode. The analysis module performs obstacle detection and analysis based on the robot's environmental information. The positioning mode switching module can execute an instruction to switch the positioning mode to a second positioning mode based on the analysis results of the analysis module.

[0028] Based on the above robot solution, the analysis module further applies a neural network model or a point cloud analysis program to perform obstacle detection and analysis.

[0029] In a third aspect, this application provides a computer-readable storage medium storing at least one instruction that is loaded and executed by a processor to implement any of the positioning control methods described in the first aspect above. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 is a basic flowchart of the positioning control method in this application;

[0032] Figure 2 is a flowchart of the specific method of step S210 in this application;

[0033] Figure 3 is a flowchart of a first specific method for step S220 in this application;

[0034] Figure 4 is a flowchart of a second specific method for step S220 in this application;

[0035] Figure 5 is a flowchart of the third specific method for step S220 in this application;

[0036] Figure 6 is a detailed flowchart of an optional positioning control method in this application. Detailed Implementation

[0037] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0040] It should also be noted that the illustrations provided in the following embodiments are merely schematic representations of the basic concept of this application. The illustrations only show elements relevant to this application and are not drawn according to the actual number, shape, and size of elements in implementation. In actual implementation, the type, quantity, and proportion of each element can be arbitrarily changed, and the element layout may be more complex. Furthermore, the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The descriptions of steps in the specification and drawings, such as "step S100," "step S200," "step S211," "step S212," etc., are used to distinguish similar steps and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0041] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0042] During the implementation of this application and the inventors' inventive research and development, the inventors discovered that some scenarios where robot RTK signals are severely interfered with or lost can be identified through environmental detection and analysis. For example, in applications involving outdoor robots, when the robot enters narrow passages or alleyways, it can be determined that signal obstruction may occur. Therefore, this solution proposes a positioning control method that can more promptly and accurately identify scenarios where RTK signals may be obstructed or where satellite positioning system data is degraded, and switch to a positioning mode that is unaffected by the obstruction. This avoids or reduces the situation where the robot's positioning mode is not switched in a timely manner, thereby improving the stability and reliability of robot operations.

[0043] Based on this, this specification provides a positioning control method in a first aspect. Figure 1 is a basic flowchart of the positioning control method, wherein step S100 is to use a first positioning mode for positioning, step S200 is to collect environmental information and perform real-time obstacle analysis based on the environmental information, and step S300 is to determine whether to switch to a second positioning mode for positioning based on the results of the real-time obstacle analysis; wherein the first positioning mode is to use at least one satellite positioning system for positioning; and the second positioning mode is to use at least one non-satellite positioning system for positioning.

[0044] In one feasible embodiment, the positioning control method corresponding to the basic flowchart of the above positioning control method is applied to a mobile device, which is a robot. The robot includes a GNSS positioning system with RTK. The first positioning mode applied by the robot is positioning based on the GNSS positioning system with RTK. Step S100 is specifically S101: positioning using a GNSS satellite positioning system with RTK. The specific implementation of step S101 is that the robot determines its own position based on the GNSS positioning system with RTK before switching positioning modes.

[0045] In another feasible embodiment based on the above embodiments, the robot includes a GNSS positioning system or a GNSS positioning system with RTK. The first positioning mode applied by the robot is positioning based at least on the GNSS positioning system or a GNSS positioning system with RTK. Furthermore, the robot also includes a VSLAM positioning system or a laser SLAM positioning system. The second positioning mode applied by the robot is positioning based at least on the VSLAM positioning system or a laser SLAM positioning system. In this embodiment, the positioning using the first positioning mode in step S100 is preferably performed by the robot before switching positioning modes, based on the GNSS positioning system or a GNSS positioning system with RTK. The TK's GNSS positioning system, VSLAM positioning system, or laser SLAM positioning system determines its own position. In summary, step S100 specifically involves S102: fusing positioning data from the satellite positioning system and non-satellite positioning systems based on a first parameter to achieve positioning using the fused data. The specific implementation of step S102 is as follows: the robot corrects the positioning data of the GNSS positioning system or the GNSS positioning system with RTK based on the positioning data from the GNSS positioning system or the GNSS positioning system with RTK, by fusing the positioning data from the VSLAM positioning system or the laser SLAM positioning system, and then performs positioning based on the corrected positioning data. The fusion process uses a Kalman filter method: the positioning data from the GNSS positioning system or the GNSS positioning system with RTK is used as observation, and the positioning data from the VSLAM positioning system or the laser SLAM positioning system is used as prior estimate; an extended Kalman filter is then used to fuse the positioning data from both systems. The first parameter means that when positioning according to the satellite positioning system, the first observation noise covariance matrix and the first process noise covariance matrix are used, wherein the trace of the first observation noise covariance matrix is ​​less than the trace of the first process noise covariance matrix. That is, when performing data fusion, it is assumed that the observation (positioning data of the satellite positioning system) error is less than the prior estimate (positioning data of the non-satellite positioning system) error, and data fusion is performed based on this.

[0046] Furthermore, based on the various positioning control methods described above, this specification provides two specific implementations of step S200: Step S210: Based on the image information in the environmental information, obstacles in the environment are detected and analyzed by analyzing the image, and it is determined whether to switch to the second positioning mode for positioning based on the position information of the obstacles; Step S220: Based on the point cloud information in the environmental information, obstacles in the environment are detected and analyzed by analyzing the point cloud, and it is determined whether to switch to the second positioning mode for positioning based on the position information of the obstacles.

[0047] Furthermore, in a feasible specific embodiment, the robot includes a VSLAM positioning system or a laser SLAM positioning system with a vision camera. The second positioning mode applied by the robot is positioning based at least on the VSLAM positioning system or the laser SLAM positioning system with a vision camera. The specific method flowchart of step S210 is shown in Figure 2. The specific method of step S210 includes S211: acquiring an image dataset of obstacles, labeling the image dataset with obstacles, and constructing and training a neural network model; S212: determining the type and location information of obstacles in the image information based on the trained neural network model; S213: If the location conditions of the obstacle satisfy any one of the first, second, third, and fourth conditions, the system switches to the second positioning mode. If the location conditions of the obstacle do not satisfy any of the first to fourth conditions, the system does not switch to the second positioning mode. The first condition is that at least one obstacle is located above the mobile device; the second condition is that at least one obstacle with a height higher than the mobile device is located within a first distance value range of the mobile device; the third condition is that at least two obstacles exist within the second distance value range of the mobile device; and the fourth condition is that at least three non-parallel obstacles exist within the third distance value range of the mobile device. The flow order of the first to fourth conditions shown in Figure 2 is only one possible implementation, but the first, second, third, and fourth conditions can be arranged in any flow order. It should be understood that the specific method steps S211, S212, and S213 described in this specific embodiment are only a feasible solution for using image processing methods to detect and analyze obstacles in the environment, and are not intended to strictly limit the method used to detect and analyze obstacles in the environment through image analysis, i.e., not to limit the scope of protection of the independent claims.

[0048] This specification further explains the specific implementation of the above-mentioned method for detecting and analyzing obstacles in the environment by analyzing images: In the process of implementing this application and in the inventor's creative research and development practice, the inventor discovered that some scenarios in which the robot's RTK signal is severely interfered with or lost mainly include scenarios such as the robot being in alleys between houses, between houses and walls, or in areas covered by forests, large trees, eaves, or awnings. To address these scenarios, the inventor discovered that the robot's environment can be detected and analyzed by analyzing environmental images. Specifically, in one feasible embodiment, the robot includes a vision sensor, and the specific implementation of step 210 is as follows: When the robot performs positioning quality analysis based on its environment, it acquires environmental images through the vision sensor, detects common obstacles that may block satellite positioning signals (such as forests, large trees, houses, walls, eaves, awnings, etc.) through image processing methods, and determines whether to switch positioning modes based on the positional relationship between the obstacles and the robot itself. Furthermore, the robot integrates an image analysis module. The specific implementation of step S211 is as follows: the robot acquires an image dataset containing the aforementioned obstacles through the image analysis module, annotates the obstacle images, and constructs semantic segmentation neural networks such as FCN, Unet, U2net, and Deeplab. The annotated dataset is used to train the neural network parameters to obtain a trained neural network model. The specific implementation of step S212 is as follows: when the robot moves, it acquires environmental images through a visual sensor and inputs them into the neural network model to detect the aforementioned common obstacles that may obstruct satellite signals and obtain the location of these obstacles in the image. The robot then acquires data from devices such as binocular cameras, structured light cameras, TOF cameras, or monocular cameras within the visual sensor. The 3D / depth image obtained by D reconstruction is used to match the feature points of the obstacles with the point cloud in the 3D / depth image to determine the three-dimensional point cloud of the obstacles; the specific implementation of step S213 is as follows: the robot determines whether its position is located in a specific area containing the obstacles based on the positional relationship between the obstacles and itself. If so, the positioning mode is switched. For example, when the eaves, awnings, forests, and tall trees are detected, it is determined whether it is under the obstacles. If so, the positioning mode is switched; when the forests, houses, and walls are detected, it is determined whether it is within a specific distance of the two-dimensional projection of the obstacles. If so, the positioning mode is switched; when multiple obstacles are detected in a specific area, it is determined whether the robot is in the area surrounded by multiple obstacles. If so, the positioning mode is switched. (4)

[0049] Furthermore, in a feasible embodiment, the robot includes a VSLAM positioning system or a laser SLAM positioning system. This specification provides a first specific method for step S220, the flowchart of which is shown in Figure 3. Step S220 specifically includes S221: determining point cloud information whose distance value from the center of the mobile device is less than a first distance value as local environment information; S222: based on the local environment information, determining whether there are obstacles whose boundary point clouds are equal to or higher than a first height value; S223: based on obstacles whose boundary point clouds are equal to or higher than the first height value, analyzing whether there are at least two obstacle boundary planes whose plane angle is less than a first angle value. If so, then switch to using the second positioning mode for positioning; if not, then do not switch to using the second positioning mode for positioning.

[0050] Based on the above embodiments, another optional embodiment is still based on the robot including a VSLAM positioning system or a laser SLAM positioning system. However, step S200 provides a second specific method in this specification. Its flowchart is shown in Figure 4, including S221: determining local environment information based on point cloud information whose distance value from the center of the mobile device is less than a first distance value; S222: determining whether there are obstacles whose boundary point clouds are equal to or higher than a first height value based on the local environment information; S224: performing two-dimensional projection of the obstacle boundary point cloud based on obstacles whose boundary point clouds are equal to or higher than the first height value; S225: analyzing whether there are at least two straight lines whose included angle is less than a first angle value based on the two-dimensional projection information. If so, switching to the second positioning mode for positioning; otherwise, not switching to the second positioning mode for positioning.

[0051] Based on the above embodiments, this specification also provides another optional embodiment, still based on the robot including a VSLAM positioning system or a laser SLAM positioning system. However, step S200 provides a third specific method in this specification, the flowchart of which is shown in Figure 5. Specifically, it includes S221: determining local environment information based on point cloud information whose distance value from the center of the mobile device is less than a first distance value; S222: determining whether there are obstacles whose boundary point clouds are equal to or higher than a first height value based on the local environment information. If yes, proceed to step S224; otherwise, do not switch to using VSLAM positioning system. Second positioning mode positioning; S224: Based on obstacles whose boundary point cloud is equal to or higher than the first height value, perform two-dimensional projection of the obstacle boundary point cloud; S226: Based on the two-dimensional projection information, analyze whether there are at least two straight lines with an included angle less than the first angle value. If so, further analyze whether the minimum distance between the at least two straight lines is less than the second distance value. If not, do not switch to using the second positioning mode positioning. If the result of analyzing whether the minimum distance between the at least two straight lines is less than the second distance value is yes, switch to using the second positioning mode positioning. If not, do not switch to using the second positioning mode positioning.

[0052] The specific implementation methods of step S220 in the above three optional embodiments are largely the same, the difference being that the analysis object selected in the method of analyzing point cloud information is slightly different. This specification further explains the specific implementation methods of the above three methods for analyzing point clouds: In the implementation process of this application and in the inventor's creative research and development practice, the inventor of this application found that although the technical solution of detecting obstacles in the environment by analyzing images can detect all common obstacles that may block satellite signals, if neural networks are used for image processing, its disadvantage is that it has high hardware performance requirements and may not be effectively deployed on some devices. In addition, the inventor found that the obstacle channel area formed by two adjacent parallel obstacles is the most common scenario where satellite positioning quality is severely degraded. Effective detection and positioning mode switching processing for the above scenario can effectively improve the stability and reliability of the robot positioning system. When the robot detects parallel obstacle channels, it can determine whether the environment is a parallel obstacle channel based on factors such as the height of the obstacle boundary, whether the obstacle boundary is parallel, and the distance between parallel obstacles.

[0053] Based on this, the positioning control methods of the above three optional embodiments can be summarized as follows: the robot acquires environmental information in real time and determines whether its position is located in a parallel obstacle channel. Here, a parallel obstacle is defined by two obstacle surfaces that have at least a specific height and can be considered parallel within a specific angle range. A parallel obstacle channel is the area between the parallel surfaces of different obstacles. To achieve the detection of the parallel obstacle channel, in the specific implementation schemes on which the above three optional embodiments rely, the robot includes a VSLAM positioning system or a laser SLAM positioning system. The specific method for acquiring environmental information in real time is through LiDAR or a vision sensor. The vision sensor can be any one or a combination of a monocular camera, a binocular camera, a structured light camera, and a TOF camera. LiDAR, binocular cameras, structured light cameras, and TOF cameras can utilize their own depth detection devices to achieve depth detection, thereby outputting a depth map or a 3D point cloud map of the environment. A monocular camera can utilize the feature point matching relationship between consecutive frames and calculate the depth of feature points based on a triangulation method, thereby obtaining a depth image or a 3D point cloud map of the environment. Therefore, based on visual sensors including but not limited to the aforementioned cameras, a 3D point cloud map of the environment can be obtained. During the robot's operation, a local 3D point cloud map within a certain range around its center is recorded. Through this local 3D point cloud map, 3D environmental information within a certain range can be obtained. In specific implementations, point cloud information within a first distance value from the robot's center can be set as local 3D environmental information. This first distance value is adjusted based on the sensitivity of the positioning system integrated into the robot. Based on the local 3D point cloud map, the parallel obstacle channel described in this application can be the area between the point clouds of two obstacle surfaces, such as the area between the point clouds corresponding to the surfaces of adjacent house walls, fences, or high-density fences. Both obstacle surface point clouds have a specific height, and the approximate plane formed by the two obstacle surface point clouds is considered approximately parallel if the included angle is less than a specific angle range. The approximate plane formed by the point cloud of the obstacle surface is determined by the fact that, since this application is based on the analysis of a 3D point cloud map of the environment, due to factors such as feature extraction and measurement errors, the obstacle boundary point cloud may not be precisely located on a single plane. However, the approximate plane formed by the obstacle boundary point cloud can be determined through data statistics, fitting, and other methods. Specifically, the approximate parallelism within a specific angle range means that, since the detection of parallel obstacle surfaces in this solution is used to determine whether it will cause RTK or GNSS signal obstruction, the approximate planes formed by the two obstacle surfaces do not need to be absolutely parallel. It can also be an angular offset with a first angle value at the parallel angle. As long as the angular offset does not exceed the first angle value, the planes can be approximately parallel when the included angle is less than a specific angle range. The specific angle range can be set based on values ​​derived by those skilled in the art based on practical experience. For example, in this embodiment, the approximate plane is set to the first angle value less than the included angle of the planes.

[0054] The specific implementation of step S221 in the above three optional embodiments is as follows: The robot detects the environment in real time through a vision sensor installed on the robot, acquires obstacle point clouds within a certain range, and forms a three-dimensional point cloud map of the environment. The three-dimensional point cloud map of the environment contains the three-dimensional coordinate information of each point. At this time, the robot can use the three-dimensional point cloud map of the environment and the three-dimensional coordinate information of the obstacle point cloud to determine the obstacle point cloud within a certain range centered on the robot as local environmental information. In the embodiments disclosed in this specification, the certain range can be specifically set by those skilled in the art based on the high-performance redundancy requirements or the requirements for high-efficiency use of hardware resources in the development process of mobile device products. One specific solution is to set the point cloud with a distance value less than a first distance value from the center of the mobile device as local environmental information.

[0055] The specific implementation of step S222 is as follows: the robot further determines whether there is an obstacle point cloud with a height equal to or greater than a specific height in the obstacle point cloud within a certain range of its own center. If so, it obtains the obstacle point cloud with a height equal to or greater than the specific height.

[0056] The specific implementation of step S223 is as follows: After the robot obtains the obstacle point cloud with a height equal to or greater than a specific height, it detects in the three-dimensional point cloud map whether there are at least two point cloud planes with a plane angle less than a first angle value. The first angle value can be set based on the experience value of parallel obstacle channels with degraded satellite positioning signals. If so, it switches to positioning using the second positioning mode.

[0057] The specific implementation of step S224 is as follows: After obtaining the obstacle point cloud with a height equal to or greater than a specific height, the obstacle point cloud is projected onto a two-dimensional plane. Since the environmental three-dimensional point cloud map contains the three-dimensional coordinate information of the obstacle point cloud, the three-dimensional coordinates are world coordinate system coordinates or can be converted to world coordinate system coordinates through coordinate transformation. At this time, the two-dimensional projection of the obstacle point cloud on the Oxy plane can be obtained by extracting the XY axis components of the coordinates.

[0058] The specific height mentioned in this embodiment can be set by those skilled in the art based on practical experience, and is generally set to the height of an obstruction that can affect general satellite positioning signals.

[0059] The specific implementation of step S225 is as follows: After projecting the obstacle point cloud onto a two-dimensional plane, the two-dimensional coordinate points can be transformed to the Hough parameter space using the Hough transform method. Each two-dimensional coordinate point corresponds to a curve in the Hough parameter space. The accumulator / grid through which the curve obtained by transforming each Oxy plane point to the Hough parameter space passes is determined using the accumulator method. The maximum value of each accumulator / grid is counted to determine the parameters of multiple straight lines. Based on the parameters of multiple straight lines, it is determined whether there are parallel lines that can be considered parallel within a specific angle range. One possible parameter is to determine whether there are at least two straight lines with an included angle less than a first angle value. If so, the second positioning mode is switched to positioning. By selecting obstacle point clouds with heights equal to or greater than a specific height for processing, this solution can save the calculation of obstacle point clouds with heights lower than the specific height, reducing computational resource consumption. By projecting the three-dimensional obstacle point cloud onto a two-dimensional plane and then performing straight line detection on the two-dimensional obstacle point cloud, this solution can avoid the model complexity caused by the three-dimensional spatial plane detection model and avoid the data volume or data complexity caused by processing three-dimensional data. Furthermore, since the obstacles dealt with in this solution are real objects from the physical world, if an object has a portion with a height equal to or greater than a certain height, it must also have a portion with a height lower than that certain height for support. Moreover, the object's horizontal position typically does not change significantly due to changes in height. Therefore, the above processing will not lead to large errors in the analysis results, but it can greatly reduce model complexity and data complexity.

[0060] The specific implementation of step S226 is based on the specific implementation of step S225, with the addition of a judgment procedure. Specifically, after determining the existence of parallel lines that can be considered parallel within a specific angle range based on the parameters of multiple straight lines, a specific solution is to further determine whether the distance between the parallel lines is less than a second distance value after determining that at least two parallel lines exist based on the angle between the lines being less than a first angle value. The second distance value can be adjusted according to the actual robot's requirements for the stability and accuracy of the positioning signal. If so, the second positioning mode is switched to be used for positioning.

[0061] The three optional embodiments provided in this specification employ a conditional detection method. This involves determining whether an obstacle point cloud with a height greater than or equal to a specific height exists within a certain range of the robot. If such an obstacle exists, parallel obstacle channel detection is performed; otherwise, it is not. This saves the robot's computational resources. Therefore, those skilled in the art can further set detection conditions based on whether the number of obstacle point clouds with a height equal to or greater than a specific height within the robot's first distance range is equal to or greater than a specific number. Furthermore, the positioning control method provided by the three optional embodiments can promptly and accurately identify the main scenarios of satellite positioning system data degradation based on the robot's local environmental information, reducing the risk of untimely positioning mode switching. Simultaneously, it eliminates the need for neural network processing, avoiding the problems of high hardware performance requirements and limited deployment applicability.

[0062] Based on the basic flowchart of the positioning control method shown in Figure 1, this specification also provides two specific method steps for step S300. One is step S301: directly using positioning data from a non-satellite positioning system to determine the current position; the other is step S302: fusing the positioning data from the satellite positioning system and the positioning data from the non-satellite positioning system based on a second parameter, and determining the current position based on the fused data. The second parameter is configured to indicate that the positioning data error of the non-satellite positioning system is less than the positioning data error of the satellite positioning system. One specific implementation of step S301 is that the robot switches to directly using positioning data from the VSLAM positioning system for positioning. For example, during the startup phase of each robot operation (GNSS-VIO system joint initialization phase), a local world coordinate system is established with the base station as the origin and the machine's initial orientation, the Earth ellipsoid normal direction, and directions perpendicular to the above two axes as coordinate axes. An RTK geocentric coordinate system (such as the WGS-84 coordinate system) is then calculated. The transformation parameters between the CGC2000 coordinate system and the local world coordinate system are used during robot operation. Based on the transformation parameters established during the initialization phase, the RTK coordinates are converted to local world coordinates. Simultaneously, the robot's coordinates in the local world coordinate system are calculated using VSLAM. When using RTK for localization, VSLAM localization is performed concurrently to calculate the robot's pose and the 3D coordinates of image feature points. The VSLAM localization data is corrected in real-time based on the RTK localization data to ensure that VSLAM localization does not accumulate errors when RTK localization is effective. When switching to localization based on VSLAM localization data, the VSLAM system continues to perform VSLAM localization calculations based on existing VSLAM localization data and the 3D coordinates of image feature points, and performs localization based on the VSLAM localization data. One specific implementation of step S302 is that when the robot switches to positioning based on the positioning data of the VSLAM positioning system, the robot can also correct the positioning data by fusing RTK data with the positioning data of the VSLAM positioning system, and perform positioning based on the corrected data. Specifically, RTK positioning data is used as observation, VSLAM positioning data is used as prior estimate, and an extended Kalman filter is used to fuse RTK positioning data and VSLAM positioning data. When positioning based on the VSLAM positioning system, a second observation noise covariance matrix is ​​used, where the trace of the second observation noise covariance matrix is ​​greater than the trace of the first process noise covariance matrix. That is, when performing data fusion, it is assumed that the observation (RTK positioning data) error is greater than the prior estimate (VSLAM positioning data) error, and data fusion is performed based on this.

[0063] To fully illustrate a feasible positioning control method, this specification provides a detailed flowchart of an optional positioning control method, as shown in Figure 6. Step S102 involves fusing fixed data from a satellite positioning system with positioning data from a non-satellite positioning system based on a first parameter, using the fused data for positioning. Step S221 involves identifying point cloud information whose distance from the center of the mobile device is less than a first distance value as local environment information. Step S222 involves determining, based on the local environment information, whether there are obstacles whose boundary point clouds are equal to or higher than a first height value. If so, proceed to step S224; otherwise, do not switch to the second positioning mode. Step S224 involves determining whether there are obstacles whose boundary point clouds are equal to or higher than the first height value. For obstacles with varying heights, a two-dimensional point cloud projection of the obstacle boundary is performed. Step S226 involves analyzing, based on the two-dimensional projection information, whether there are at least two straight lines with an included angle less than a first angle value. If so, further analysis is performed to determine if the minimum distance between the at least two straight lines is less than a second distance value. If not, the second positioning mode is not used for positioning. If the result of analyzing whether the minimum distance between the at least two straight lines is less than the second distance value is yes, the second positioning mode is used for positioning; otherwise, the second positioning mode is not used for positioning. Step S302 involves fusing the positioning data from the satellite positioning system and the positioning data from the non-satellite positioning system based on a second parameter, and determining the current position based on the fused data. The second parameter is configured to indicate that the positioning data error of the non-satellite positioning system is less than the positioning data error of the satellite positioning system. It should be understood that step S102 shown in this embodiment and Figure 6 can be replaced by step S101, and steps S221, S222, S224, and S226 can be replaced by steps S211, S212, and S213 or steps S221, S222, and S223 or steps S221, S222, S224, and S225 corresponding to the above optional embodiments. Step S302 can be replaced by step S301.

[0064] In specific implementations of the above embodiments, the satellite positioning system and / or non-satellite positioning system included in the mobile device or the robot also include an IMU and / or a wheeled odometer as a method to assist in improving positioning accuracy.

[0065] This specification provides a robot in a second aspect. The robot is a mobile device and includes a satellite positioning system, a non-satellite positioning system, an analysis module, and a positioning mode switching module. The robot performs positioning based on the satellite positioning system or based on a combination of the satellite positioning system and the non-satellite positioning system in a first positioning mode. The analysis module performs obstacle detection and analysis based on the robot's environmental information. The positioning mode switching module can execute an instruction to switch the positioning mode to a second positioning mode based on the analysis results of the analysis module.

[0066] Furthermore, the analysis module applies a neural network model or a point cloud analysis program to perform obstacle detection and analysis.

[0067] This specification provides, in a third aspect, a computer-readable storage medium storing at least one instruction that is loaded and executed by a processor to implement any of the positioning control methods described in the first aspect above.

[0068] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the descriptions of the embodiments described later are relatively simple, and relevant parts can be referred to the descriptions of the foregoing embodiments.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A positioning control method, applied to a mobile device including at least two positioning systems, characterized in that, The positioning control method includes: Use the first positioning mode for positioning; Collect environmental information and perform real-time obstacle analysis based on that information; Determine whether to switch to the second positioning mode based on the results of real-time obstacle analysis; The first positioning mode is based on at least one satellite positioning system, and the second positioning mode is based on at least one non-satellite positioning system.

2. The positioning control method according to claim 1, characterized in that, The first positioning mode includes: Positioning is achieved directly using positioning data from the satellite positioning system; Alternatively, positioning data from satellite positioning systems and positioning data from non-satellite positioning systems can be fused based on the first parameter to achieve fused data positioning.

3. The positioning control method according to claim 1, characterized in that, The real-time obstacle analysis specifically includes: Based on the image information in the environmental information, obstacles in the environment are detected by analyzing the images, and it is determined whether to switch to the second positioning mode for positioning based on the location information of the obstacles.

4. The positioning control method according to claim 3, characterized in that, The method of detecting obstacles in the environment by analyzing images includes: Obtain an image dataset of obstacles, annotate the image dataset with obstacles, and construct and train a neural network model; The type and location information of obstacles in the image information are determined based on the trained neural network model; If the location conditions of the obstacle satisfy any one of the first, second, third, and fourth conditions, then the positioning mode is switched to use the second positioning mode. The first condition is that at least one obstacle is located above the mobile device; the second condition is that at least one obstacle with a height higher than the mobile device is located within the first distance value range of the mobile device; the third condition is that there are at least two obstacles within the second distance value range of the mobile device; and the fourth condition is that there are at least three non-parallel obstacles within the third distance value range of the mobile device.

5. The positioning control method according to claim 1, characterized in that, The real-time obstacle analysis specifically includes: Based on the point cloud information in the environmental information, obstacles in the environment are detected by analyzing the point cloud, and it is determined whether to switch to the second positioning mode for positioning based on the position information of the obstacles.

6. The positioning control method according to claim 5, characterized in that, The method of detecting and analyzing obstacles in the environment by analyzing point clouds includes: Point cloud information whose distance value from the center of the mobile device is less than a first distance value is determined as local environment information; Based on the local environmental information, analyze whether there are obstacles whose boundary point clouds are equal to or higher than the first height value.

7. The positioning control method according to claim 5, characterized in that, The method of detecting and analyzing obstacles in the environment by analyzing point clouds also includes: Based on the boundary point cloud of obstacles that are equal to or higher than the first height value, analyze whether there are at least two obstacle boundary planes with a plane angle smaller than the first angle value. If so, switch to using the second positioning mode for positioning.

8. The positioning control method according to claim 5, characterized in that, The method of detecting and analyzing obstacles in the environment by analyzing point clouds also includes: Based on obstacles whose boundary point cloud is equal to or higher than the first height value, perform two-dimensional projection of the obstacle boundary point cloud; Based on the two-dimensional projection, analyze whether there are at least two straight lines with an included angle less than the first angle value. If so, switch to the second positioning mode for positioning.

9. The positioning control method according to claim 5, characterized in that, The method of detecting and analyzing obstacles in the environment by analyzing point clouds also includes: Based on obstacles whose boundary point cloud is equal to or higher than the first height value, perform two-dimensional projection of the obstacle boundary point cloud; Based on the two-dimensional projection, it is analyzed whether there are at least two straight lines with an included angle less than the first angle value. If so, it is further analyzed whether the minimum distance between the at least two straight lines is less than the second distance value. If so, it is switched to the second positioning mode for positioning.

10. The positioning control method according to claim 1, characterized in that, The switching to the second positioning mode includes: The current location is determined directly using the positioning data from the non-satellite positioning system.

11. The positioning control method according to claim 1, characterized in that, The switching to the second positioning mode includes: Based on the second parameter, the positioning data of the satellite positioning system and the positioning data of the non-satellite positioning system are fused, and the current location is determined based on the fused data. The second parameter is configured to indicate that the positioning data error of the non-satellite positioning system is less than the positioning data error of the satellite positioning system.

12. The positioning control method according to claim 1, characterized in that, The satellite positioning system includes a GNSS positioning system or a GNSS positioning system with RTK.

13. The positioning control method according to claim 1, characterized in that, The non-satellite positioning system is a visual camera system, a VSLAM system, or a laser SLAM system.

14. The positioning control method according to claim 12 or 13, characterized in that, The satellite positioning system and / or the non-satellite positioning system also have an IMU and / or a wheeled odometer.

15. A robot, said robot being a mobile device, characterized in that, The robot includes a satellite positioning system, a non-satellite positioning system, an analysis module, and a positioning mode switching module; The robot performs positioning based on a satellite positioning system or based on a combination of a satellite positioning system and a non-satellite positioning system in the first positioning mode. The analysis module performs obstacle detection and analysis based on the robot's environmental information; The positioning mode switching module can execute an instruction to switch the positioning mode to positioning based on the analysis results of the analysis module.

16. The robot according to claim 15, characterized in that, The analysis module uses a neural network model or a point cloud analysis program to detect and analyze obstacles.

17. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the positioning control method as described in any one of claims 1-14.

Images