Robot positioning method, device and equipment and storage medium

Abstract

The invention relates to the technical field of robot position measurement, in particular to a robot positioning method, device and equipment and a storage medium, and the method comprises the steps: carrying out the data fusion of robot satellite positioning data and robot motion data, and obtaining robot positioning information; performing feature detection on the binocular environment image to obtain target environment feature information; generating a feature map based on the robot positioning information and the target environment feature information; and determining the target position of the robot based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map. According to the invention, the robustness and accuracy of the robot positioning process can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot position measurement, and in particular to a robot positioning method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous development of the robot industry, many robots for serving people's life have begun to enter people's daily life, such as companion robots and outdoor work robots. In order to control the robot to accurately travel according to the predetermined route, so as to complete the task in the process of traveling, the position of the robot in the environment where it is located needs to be accurately determined, that is, the positioning of the robot needs to be realized.

[0003] At present, one way to realize robot positioning is to preset a base station near the predetermined working area of the robot. The base station is usually erected on a point with known accurate coordinates. The base station calculates its own positioning result in real time by receiving satellite signals, and compares it with the known coordinates to obtain an error correction value (RTK difference information). The base station sends the error correction value to the nearby robot in real time through wireless communication (such as radio, 4G / 5G network, etc.). The robot receives satellite signals and correction values sent by the base station at the same time, and calculates its own coordinates by combining the two.

[0004] However, first, the signal coverage range of the base station is limited. Once the position of the robot exceeds the signal range of the base station, the base station cannot send the difference information to the robot to realize the positioning of the robot. Second, the signal stability of the base station is poor. When the signal is unstable, the RTK difference accuracy error is large and invalid, which further leads to difficulty in realizing real-time and accurate positioning of the robot. It can be seen that the robustness and accuracy of the current robot positioning process are poor. SUMMARY

[0005] In order to improve the robustness and accuracy of the robot positioning process, the present application provides a robot positioning method, device, equipment and storage medium.

[0006] In a first aspect, the present application provides a robot positioning method, comprising:

[0007] fusing robot satellite positioning data and robot motion data to obtain robot positioning information;

[0008] detecting features of the binocular environment image to obtain target environment feature information;

[0009] generating a feature map based on the robot positioning information and the target environment feature information;

[0010] determine a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0011] In a second aspect, the present application provides a robot positioning device, comprising:

[0012] a data fusion module configured to perform data fusion on the robot satellite positioning data and the robot motion data to obtain robot positioning information;

[0013] a feature detection module configured to perform feature detection on the binocular environment image to obtain target environment feature information;

[0014] a map generation module configured to generate a feature map based on the robot positioning information and the target environment feature information;

[0015] a position determination module configured to determine a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0016] In a third aspect, the present application provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0017] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0018] In a fifth aspect, the present application further provides a computer program product. The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of any of the above method embodiments.

[0019] The robot positioning method, device, equipment and storage medium described above, through data fusion of the robot satellite positioning data and the robot motion data, obtain robot positioning information; feature detection is performed on the binocular environment image to obtain target environment feature information; a feature map is generated based on the robot positioning information and the target environment feature information; and a robot target position is determined based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map. Through the above implementation, first, the application gets rid of the limitation of the base station used in the positioning of the robot in the traditional technology, so as to eliminate the limitation of the signal coverage range on the robot positioning. In addition, the robot itself can collect the robot satellite positioning data and the robot motion data without the participation of the base station, so as to eliminate the problem of poor robot positioning accuracy caused by unstable base station signals. Second, by fusing the robot satellite positioning data and the robot motion data of different dimensions, the defects of single-dimensional data can be eliminated, and the advantages of multi-dimensional data information can be fully played, so as to improve the accuracy of the calculated robot positioning information, and then improve the accuracy of the subsequently calculated robot target position. Third, the target environment feature information obtained by feature detection on the binocular environment image can be used to accurately reflect the environment in which the robot is located, so as to improve the accuracy of the subsequently determined robot position in the environment. Fourth, the application gets rid of the dependence on the feature perception of the laser radar LIDAR, effectively simplifies the engineering complexity, and facilitates the lightweight development of the computing platform. As can be seen from the above, the scheme shown in the application can greatly improve the robustness and accuracy of robot positioning.

[0020] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the application, nor is it used to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0022] Figure 1 A robot positioning method flowchart provided in the embodiments of the application;

[0023] Figure 2 A schematic diagram of a feature detection network processing a binocular environment image provided in the embodiments of the application;

[0024] Figure 3 An initial three-dimensional environment schematic diagram provided in an embodiment of the present application;

[0025] Figure 4 A structural schematic diagram of a robot positioning device provided in an embodiment of the present application;

[0026] Figure 5 A structural schematic diagram of a computer device provided in an embodiment of the present application;

[0027] Figure 6 An internal structure diagram of a computer readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0029] It should be noted that the terms "first", "second" and the like in the description, claims and above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0030] In this paper, the term "and / or" is only a description of the relationship between the associated objects, which means that there can be three relationships. For example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally means that the front and rear associated objects are a "or" relationship.

[0031] Embodiment one

[0032] Figure 1 A robot positioning method flowchart provided in embodiment one of the present application, with reference to Figure 1 The method can be executed by a device executing the method, which can be realized by software and / or hardware, and the method comprises:

[0033] S110, data fusion is performed on the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0034] With the continuous progress of robot technology, various kinds of robots for liberating human labor have entered human daily life; for example, robots include but are not limited to outdoor mowing robots for mowing lawns, outdoor transport robots for transporting goods, and outdoor inspection robots for performing equipment or area inspection, etc.; in order to facilitate the robot to operate automatically according to the set route to maximize the liberation of human labor, it is necessary to accurately position the robot in the environment where it is located, that is, the positioning of the robot needs to be realized first.

[0035] It should be noted that the robot provided in the embodiment is provided with a satellite communication module, which is used to communicate with a positioning satellite to obtain the position data of the robot measured by the positioning satellite, and the position data of the robot measured by the positioning satellite is recorded as robot satellite positioning data. It should also be noted that the accuracy of the position data of the robot measured by the positioning satellite is low, and if only the robot satellite positioning data is used as the robot positioning data, it is difficult to ensure the accuracy of the subsequent control of the robot moving according to the set route.

[0036] In order to improve the accuracy of the robot positioning data, the embodiment also collects data of other dimensions in addition to the robot satellite positioning data, and performs data fusion with the robot satellite positioning data, so as to improve the dimension of the original data used to determine the robot positioning data, so as to improve the comprehensiveness of the original data used to determine the robot positioning data, thereby greatly improving the accuracy of the calculated robot positioning data. Therefore, the embodiment also provides a motion sensor on the robot, which is used to collect robot motion data generated when the robot is moving, such as three-axis acceleration, three-axis angular velocity, running mileage, etc., which is not limited. In other embodiments, other kinds of original data collectors are also provided on the robot, and the data obtained by the original data collector is used as one dimension of the original data required for data fusion; for example, the other kinds of original data collectors also include one or more of latitude and longitude sensors, altitude sensors, laser radars and cameras, which are not limited.

[0037] In the embodiment, by performing data fusion on the robot satellite positioning data and the robot motion data, the robot positioning information can be obtained, and it should be noted that the accuracy of the robot positioning information is greatly improved compared with the robot positioning data determined by single-dimensional original data, which not only facilitates to improve the positioning accuracy of the subsequent determined positioning data of the robot in the environment where it is located, but also improves the accuracy of the subsequent control of the robot moving according to the set route.

[0038] S120, feature detection is performed on the binocular environment image to obtain target environment feature information.

[0039] It should be noted that the embodiment aims to determine the positioning data of the robot in the environment where it is located, and for this purpose, a corresponding environment map of the environment needs to be constructed. In order to improve the construction speed of the environment map, the embodiment adopts an image processing method to construct a lightweight environment map. For this purpose, the embodiment also provides a binocular camera on the robot, which is used to collect images of the environment where the robot is located, and the images of the environment where the robot is located are denoted as binocular environment images. The binocular environment images include a left eye environment image (Left Image) corresponding to a left camera of the binocular camera, and a right eye environment image (Right Image) corresponding to a right camera of the binocular camera.

[0040] In order to construct a lightweight environment map based on the binocular environment image, a pre-set deep learning model (DNN, Deep Neural Network) can be used to detect the features of the collected binocular environment image, so as to obtain the feature information of each object in the environment corresponding to the binocular environment image. For example, the feature information includes but is not limited to point features, line features and surface features on each object, and can cover one-dimensional features (point features), two-dimensional features (line features) and three-dimensional features (geometric surface features) of the object. The feature information corresponding to each object in the environment in the binocular environment image is denoted as target environment feature information, and the lightweight environment map can be constructed through the target environment feature information.

[0041] S130, generating a feature map based on the robot positioning information and the target environment feature information.

[0042] In order to construct the above-mentioned lightweight environment map, the robot positioning information obtained in the S110 step is also needed in the embodiment. Through the robot positioning information and the target environment feature information, a lightweight environment map can be constructed, and the lightweight environment map constructed through the robot positioning information and the target environment feature information is denoted as a feature map. It should be noted that since the robot positioning information is also introduced in the process of constructing the feature map, the feature map can also reflect the motion trajectory of the robot.

[0043] It should be noted that in the prior art, the construction of the environment map often relies heavily on the environment point cloud data collected by the laser radar (LIDAR) to construct the environment map, but the efficiency of constructing the environment map by this scheme is far lower than the image processing method (extracting target environment feature information from the collected binocular environment image, and constructing a feature map based on the robot positioning information and the target environment feature information) adopted in the present application. Compared with the prior art, the present application can realize the lightweight construction of the environment map, thereby effectively improving the construction efficiency of the environment map.

[0044] S140, determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0045] Among them, the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image and the target environment feature information can be used as a data set to reflect the environment where the robot is at the current time, and the motion trajectory of the robot embodied by the feature map can be used to reflect the specific position of the robot at the current time. By combining the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map, the current position of the robot in the environment it is in can be accurately calculated, and this position is recorded as the robot target position.

[0046] It should be noted that the embodiment fuses the robot satellite positioning data and the robot motion data to obtain robot positioning information; detects features of the binocular environment image to obtain target environment feature information; generates a feature map based on the robot positioning information and the target environment feature information; and determines a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map. Through the above implementation, first, the application eliminates the limitation of the base station used in the positioning of the robot in the traditional technology, so as to eliminate the limitation of the signal coverage range on the robot positioning. In addition, the robot itself can collect the robot satellite positioning data and the robot motion data without the participation of the base station, so as to eliminate the problem of poor robot positioning accuracy caused by unstable base station signals. Second, by fusing the robot satellite positioning data and the robot motion data of different dimensions, the defects of single-dimensional data can be eliminated, and the comprehensive advantages of multi-dimensional data information can be played, so as to improve the accuracy of the calculated robot positioning information, and then improve the accuracy of the subsequently calculated robot target position. Third, the target environment feature information obtained by the feature detection of the binocular environment image can be used to accurately reflect the environment in which the robot is located, so as to improve the accuracy of the subsequently determined robot position in the environment. Fourth, the application eliminates the dependence on the feature perception of the laser radar LIDAR, effectively simplifies the engineering complexity, and facilitates the lightweight development of the computing platform. As can be seen from the above, the scheme of the application can greatly improve the robustness and accuracy of the robot positioning.

[0047] Embodiment two

[0048] The robot positioning method provided in the second embodiment of the application optimizes the "fusing the robot satellite positioning data and the robot motion data to obtain robot positioning information" in the first embodiment. It should be noted that the parts not described in this embodiment can be referred to the description of other embodiments. The robot motion data at least includes inertial measurement data and motion mileage data. Correspondingly, the method comprises:

[0049] S211, at least fusing the robot satellite positioning data, the inertial measurement data, and the motion mileage data to obtain a robot initial position and a robot initial travel distance.

[0050] It should be noted that although the robot satellite positioning data can be used for robot positioning, the positioning accuracy is poor and cannot meet the actual use requirements. In order to improve the positioning accuracy of the robot, the embodiment adopts a data fusion manner to determine robot positioning information with higher accuracy. The data required for data fusion in the embodiment specifically includes robot satellite positioning data, inertial measurement data and motion mileage data, which is not specifically limited in other embodiments.

[0051] In order to collect robot satellite positioning data, the embodiment pre-sets a satellite communication module in the robot, which is used for communicating with a positioning satellite to obtain robot satellite positioning data. In order to collect inertial measurement data, the embodiment pre-sets an inertial measurement unit (IMU) in the robot, which is used to obtain inertial measurement data of the robot. In the embodiment, the inertial measurement data specifically includes three-axis acceleration data and / or three-axis angular velocity data of the robot, which is not specifically limited in other embodiments. In order to collect motion mileage data, the embodiment pre-sets an odometer in the robot, which is used to obtain the mileage of the trajectory walked by the robot from the starting point to the current time.

[0052] Specifically, the robot satellite positioning data, the inertial measurement data and the motion mileage data collected by the robot at the current time are all input into the corresponding data fusion calculation formula for calculation, so as to obtain the initial position P0 of the robot and the initial travel distance D0 of the robot. The initial position of the robot is used to represent the position of the robot obtained by improving the accuracy of the robot satellite positioning data based on the robot satellite positioning data at the current time. The initial travel distance of the robot is used to represent the length of the trajectory walked by the robot from the starting point to the current time.

[0053] S212, obtaining robot positioning information based on the initial position of the robot and the initial travel distance of the robot.

[0054] The robot positioning information includes the initial position P0 of the robot and the initial travel distance D0 of the robot.

[0055] S220, performing feature detection on the binocular environment image to obtain target environment feature information.

[0056] S230, generating a feature map based on the robot positioning information and the target environment feature information.

[0057] S240, determining a target position of the robot based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map.

[0058] Embodiment three

[0059] The robot positioning method provided in Embodiment three optimizes the "performing feature detection on binocular environment images to obtain target environment feature information" in Embodiment one; it should be noted that the parts not described in detail in this embodiment can be referred to the descriptions in other embodiments. This method comprises the following steps.

[0060] S310, performing data fusion on the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0061] S321, processing the binocular environment images based on the feature detection network to obtain target environment point features and target environment line features.

[0062] It should be noted that this embodiment aims to determine the accurate position of the robot at the current time in the environment in which it is located, and for this purpose, the environment information of the environment in which the robot is located at the current time also needs to be determined, which reflects the features of the corresponding environment. This embodiment determines the environment information by processing the collected environment images, and for this purpose, the robot is preconfigured with a camera, which is specifically a binocular camera in this embodiment. The binocular camera is used to collect environment images, and the environment images collected by the binocular camera are denoted as binocular environment images.

[0063] To determine the environment information from the binocular environment images, this embodiment is preconfigured with a feature detection network, which is a trained detection network, and is used to analyze the binocular environment images to determine the features of the corresponding environment. It should be noted that the environment features initially determined by the feature detection network specifically include: target environment point features and target environment line features; wherein the target environment point features are the feature points of objects in the environment within a certain range of the robot, and the target environment line features are the feature lines of objects in the environment within a certain range of the robot.

[0064] S322, processing the target environment point features, the target environment line features, target environment point feature position information, and target environment line feature position information based on a visual model to obtain target environment surface features.

[0065] The target environment point feature and the target environment line feature are one-dimensional features and two-dimensional features respectively, and the environment in which the robot is located is a three-dimensional feature, so if a three-dimensional feature is further generated based on the target environment point feature and the target environment line feature, the three-dimensional environment in which the robot is located can be further accurately reflected, thereby improving the positioning accuracy of the robot in the three-dimensional environment. To this end, the embodiment determines position information of the target environment point feature in the space in which the environment is located, denoted as target environment point feature position information, and determines position information of the target environment line feature in the space in which the environment is located, denoted as target environment line feature position information. Then, the target environment point feature, the target environment line feature, the target environment point feature position information and the target environment line feature position information are processed by a preset feature processing model, and a three-dimensional feature can be obtained. The three-dimensional feature can reflect the geometric surface features of objects in the environment, and the geometric surface features are denoted as target environment surface features.

[0066] S323, obtaining target environment feature information based on the target environment point feature, the target environment line feature and the target environment surface feature.

[0067] The target environment feature information includes the target environment point feature F1, the target environment line feature F2 and the target environment surface feature F3.

[0068] S330, generating a feature map based on the robot positioning information and the target environment feature information.

[0069] S340, determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map.

[0070] Embodiment four

[0071] The robot positioning method provided in the fourth embodiment of the application optimizes the "processing the binocular environment image based on the feature detection network to obtain the target environment point feature and the target environment line feature" in the third embodiment. It should be noted that the parts not described in detail in this embodiment can be referred to the descriptions of other embodiments. The method comprises:

[0072] S410, fusing the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0073] S421A, processing the binocular environment image based on the feature detection network to obtain initial environment point features and initial environment line features.

[0074] The binocular environment image includes a left eye environment image (Left Image) and a right eye environment image (Right Image). In order to facilitate the extraction of environment features from the left eye environment image and the right eye environment image, a feature detection network trained is preset in the embodiment. Referring to Figure 2 The left eye environment image (Left Image) and the right eye environment image (Right Image) are input into the feature detection network for feature extraction, so as to obtain environment point features and environment line features. The environment point features are recorded as initial environment point features, and the environment line features are recorded as initial environment line features.

[0075] S421B, the initial environment point features and the initial environment line features are respectively subjected to position correction processing to obtain target environment point features and target environment line features.

[0076] It should be noted that the initial environment point features are used to describe a feature point on an object in the environment in which the robot is located. However, in actual operation, it is found that the positions of the initial environment point features and the initial environment line features output by the feature detection network may deviate from their true positions. Since the target environment point features and the target environment line features are subsequently used not only to calculate environment surface features but also to participate in the construction of an environment map, if the positions of the initial environment point features and the initial environment line features deviate from their true positions, the accuracy of the environment surface features and the environment map will decrease. Therefore, the initial environment point features and the initial environment line features need to be subjected to position correction. The embodiment provides a feature position correction model for correcting the positions of the initial environment point features and the initial environment line features. The initial environment point features and the initial environment line features that are corrected by the feature position correction model are recorded as target environment point features and target environment line features, respectively.

[0077] S422, the target environment point features, the target environment line features, target environment point feature position information, and target environment line feature position information are processed based on a vision model to obtain target environment surface features.

[0078] S423, target environment feature information is obtained based on the target environment point features, the target environment line features, and the target environment surface features.

[0079] S430, a feature map is generated based on the robot positioning information and the target environment feature information.

[0080] S440, a robot target position is determined based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0081] Embodiment five

[0082] The robot positioning method provided in the fifth embodiment of the present application optimizes the "determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map" in the first embodiment. It should be noted that the parts not described in detail in the present embodiment can be referred to the descriptions of other embodiments. The method comprises the following steps:

[0083] S510, data fusion is performed on the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0084] S520, feature detection is performed on the binocular environment image to obtain target environment feature information.

[0085] S530, a feature map is generated based on the robot positioning information and the target environment feature information.

[0086] S541, an initial three-dimensional environment is constructed based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, and the target environment feature information.

[0087] To determine the specific position of the robot in the environment, the environment where the robot is located needs to be determined first. The robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, and the target environment feature information obtained through the above steps can be used to construct a virtual three-dimensional environment. In the present embodiment, a three-dimensional environment generation model is preset to process the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, and the target environment feature information to generate a virtual three-dimensional environment corresponding to the environment where the robot is located. This virtual three-dimensional environment is referred to as an initial three-dimensional environment. For example, the initial three-dimensional environment is as shown in FIG. 5. Figure 3

[0088] S542, a robot target position is determined based on the initial three-dimensional environment and the feature map.

[0089] The initial three-dimensional environment and the feature map obtained through the above steps can be matched to determine the accurate position of the robot in the environment, which is referred to as the robot target position.

[0090] Embodiment six

[0091] ​The robot positioning method provided in the embodiment six of the present application supplements the steps after "fusing the robot satellite positioning data and the robot motion data to obtain robot positioning information" in the embodiment one; it needs to be noted that, the parts not described in detail in the present embodiment can be referred to the descriptions in other embodiments, and the method comprises the following steps:

[0092] S610, fusing the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0093] S620, performing feature detection on the binocular environment image to obtain target environment feature information.

[0094] S630, generating a feature map based on the robot positioning information and the target environment feature information.

[0095] S640, determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information and the feature map.

[0096] S650, constructing a map boundary based on the robot positioning information and the feature map.

[0097] In the case of determining the robot target position in the environment, the robot needs to be controlled to move in the environment subsequently, in order to prevent the robot from moving out of the environment, the robot positioning information and the feature map obtained by the preset boundary construction tool can be processed to calculate the boundary of the environment, and the boundary of the environment is recorded as the map boundary.

[0098] Specifically, the robot initial travel distance in the robot positioning information is obtained, and the robot initial travel distance in the present embodiment is recorded as d safe The feature map is further analyzed to determine the obstacle sequence O and the path point sequence P, wherein, The coordinate information of m obstacles is contained in O, and the coordinate information of n path points is contained in P.

[0099] Further, the path tangent vector and the path normal vector are calculated, wherein:

[0100]

[0101] Then, the obstacle influence function w kj is calculated, wherein:

[0102] σ is the influence range parameter.

[0103] Then, a boundary offset is calculated, the boundary offset includes a left boundary offset and a right boundary offset in the present application and the right boundary offset wherein:

[0104]

[0105] wherein, α and β are adjustment parameters.

[0106] Further, a boundary point is calculated, the boundary point includes a left boundary point L k and a right boundary point R k wherein:

[0107]

[0108]

[0109] Finally, a boundary smoothing process is performed on the above boundary points by using a cubic spline interpolation method to obtain a map boundary, and the boundary smoothing process formula is as follows:

[0110]

[0111] Embodiment seven

[0112] The robot positioning method provided in the embodiment seven of the present application supplements the steps after "determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map" in the embodiment six; it should be noted that the parts not described in detail in the present embodiment can be referred to the descriptions of other embodiments, and the method comprises the following steps:

[0113] S710, data fusion is performed on the robot satellite positioning data and the robot motion data to obtain robot positioning information.

[0114] S720, feature detection is performed on the binocular environment image to obtain target environment feature information.

[0115] S730, a feature map is generated based on the robot positioning information and the target environment feature information.

[0116] S740, a robot target position is determined based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0117] S750, a map boundary is constructed based on the robot positioning information and the feature map.

[0118] S760, set a robot travel path based on the feature map and the map boundary.

[0119] The feature map can be used to show a running track of the robot, the robot travel path can be set on the feature map within the map boundary, and the robot travel path can be used to control the robot to move according to the robot travel path after the robot travel path is set.

[0120] S770, control the robot to move based on the robot travel path.

[0121] It should be understood that, although each step in the flowchart involved in each embodiment described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0122] Embodiment Eight

[0123] Based on the same inventive concept, the present embodiment also provides a robot positioning device for implementing the robot positioning method described above. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more robot positioning device embodiments provided below can refer to the limitations of the robot positioning method described above, which will not be repeated here.

[0124] In the present embodiment, as shown in Figure 4 a robot positioning device is provided, comprising:

[0125] a data fusion module configured to fuse robot satellite positioning data and robot motion data to obtain robot positioning information;

[0126] a feature detection module configured to detect features of a binocular environment image to obtain target environment feature information;

[0127] a map generation module configured to generate a feature map based on the robot positioning information and the target environment feature information;

[0128] A position determining module is configured to determine a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map.

[0129] Each of the above modules can be implemented by software, hardware, or a combination thereof, in whole or in part. Each of the above modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be invoked and executed by a processor to perform operations corresponding to each of the above modules.

[0130] It should be noted that, in the embodiment, the robot satellite positioning data and the robot motion data are fused to obtain the robot positioning information; the binocular environment image is feature-detected to obtain the target environment feature information; the robot positioning information and the target environment feature information are used to generate the feature map; and the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment image, the target environment feature information, and the feature map are used to determine the robot target position. Through the above implementation, first, the application is free from the restriction of a base station in the positioning of a robot in the prior art, so that the limitation of a signal coverage range on robot positioning can be eliminated. In addition, the robot itself can collect the robot satellite positioning data and the robot motion data without the participation of the base station, so that the problem of poor robot positioning accuracy caused by unstable base station signals can be eliminated. Second, the robot satellite positioning data and the robot motion data of different dimensions are fused, so that the defects of single-dimensional data can be eliminated, and the comprehensive advantages of multi-dimensional data information can be exerted, thereby improving the accuracy of the calculated robot positioning information and the accuracy of the subsequently calculated robot target position. Third, the target environment feature information obtained by feature detection of the binocular environment image can be used to accurately reflect the environment in which the robot is located, so as to improve the accuracy of the subsequently determined robot position in the environment. Fourth, the application is free from the feature perception dependence on a laser radar (LIDAR) in the prior art, effectively simplifying the engineering complexity, and facilitating the lightweight development of a computing platform. As can be seen from the above, the scheme of the application can greatly improve the robustness and accuracy of robot positioning.

[0131] In an optional embodiment, the robot motion data at least includes inertial measurement data and motion mileage data; and in the fusion of the robot satellite positioning data and the robot motion data to obtain the robot positioning information, the data fusion module is specifically configured to:

[0132] at least fuse the robot satellite positioning data, the inertial measurement data, and the motion mileage data to obtain a robot initial position and a robot initial travel distance;

[0133] Obtain robot positioning information based on the robot initial position and the robot initial travel distance.

[0134] In an optional embodiment, in feature detection on binocular environment images to obtain target environment feature information, the feature detection module is specifically configured to:

[0135] Process the binocular environment images based on the feature detection network to obtain target environment point features and target environment line features;

[0136] Process the target environment point features, the target environment line features, target environment point feature position information, and target environment line feature position information based on the visual model to obtain target environment surface features;

[0137] Obtain target environment feature information based on the target environment point features, the target environment line features, and the target environment surface features.

[0138] In an optional embodiment, in processing binocular environment images based on a feature detection network to obtain target environment point features and target environment line features, the feature detection module is specifically configured to:

[0139] Process the binocular environment images based on the feature detection network to obtain initial environment point features and initial environment line features;

[0140] Respectively correct the positions of the initial environment point features and the initial environment line features to obtain target environment point features and target environment line features.

[0141] In an optional embodiment, in determining a robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment images, the target environment feature information, and the feature map, the position determination module is specifically configured to:

[0142] Construct an initial three-dimensional environment based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environment images, and the target environment feature information.

[0143] Determine a robot target position based on the initial three-dimensional environment and the feature map.

[0144] In an optional embodiment, the robot positioning device further comprises:

[0145] A boundary construction module configured to construct a map boundary based on the robot positioning information and the feature map.

[0146] In an optional embodiment, the robot positioning apparatus further comprises:

[0147] a path setting module, configured to set a robot travel path based on the feature map and the map boundary;

[0148] a motion control module, configured to control robot motion based on the robot travel path.

[0149] Embodiment Nine

[0150] In this embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 5 The computer device comprises a processor, a memory and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a robot positioning method.

[0151] Those skilled in the art can understand that Figure 5 the structure shown in the above

[0152] Embodiment Ten

[0153] In this embodiment, a computer readable storage medium is provided, as shown in Figure 6 on which a computer program is stored. The computer program is executed by the processor to implement the steps in each of the above method embodiments.

[0154] Embodiment Eleven

[0155] In this embodiment, a computer program product is provided, comprising a computer program. The computer program is executed by the processor to implement the steps in each of the above method embodiments.

[0156] It should be noted that the collected information is information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of the country and region, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user authorization or refusal.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present disclosure can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present disclosure can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0158] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present disclosure.

[0159] The above-described embodiments are merely illustrative of several embodiments of the present disclosure, which are described in a relatively specific and detailed manner, but should not be construed as limiting the scope of the patent of the present disclosure. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the appended claims.

Claims

1. A robot localization method, characterized in that, include: By fusing robot satellite positioning data and robot motion data, robot positioning information is obtained. Feature detection is performed on binocular environmental images to obtain target environmental feature information; Based on the robot's localization information and the target environment's feature information, a feature map is generated; The robot target position is determined based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environmental image, the target environmental feature information, and the feature map.

2. The method according to claim 1, characterized in that, The robot motion data includes at least inertial measurement data and motion distance data; correspondingly, the fusion of robot satellite positioning data and robot motion data to obtain robot positioning information includes: At least the robot's satellite positioning data, inertial measurement data, and motion distance data should be fused to obtain the robot's initial position and initial travel distance. Based on the robot's initial position and initial travel distance, the robot's positioning information is obtained.

3. The method according to claim 1, characterized in that, The step of performing feature detection on the binocular environment image to obtain target environment feature information includes: The binocular environment image is processed based on the feature detection network to obtain the target environment point features and the target environment line features; Based on the visual model, the target environment point features, the target environment line features, the target environment point feature location information, and the target environment line feature location information are processed to obtain the target environment surface features; Based on the target environment point features, the target environment line features, and the target environment surface features, target environment feature information is obtained.

4. The method according to claim 3, characterized in that, The feature detection network processes the binocular environment image to obtain target environment point features and target environment line features, including: The binocular environment image is processed based on the feature detection network to obtain the initial environment point features and the initial environment line features; The initial environmental point features and the initial environmental line features are respectively subjected to position correction processing to obtain the target environmental point features and the target environmental line features.

5. The method according to claim 1, characterized in that, The process of determining the robot target position based on the robot's satellite positioning data, robot motion data, robot positioning information, binocular environmental image, target environmental feature information, and feature map includes: Based on the robot's satellite positioning data, robot motion data, robot positioning information, binocular environmental image, and target environmental feature information, an initial three-dimensional environment is constructed; Based on the initial 3D environment and the feature map, the robot target position is determined.

6. The method according to claim 1, characterized in that, After fusing robot satellite positioning data and robot motion data to obtain robot positioning information, the process further includes: The map boundary is constructed based on the robot's positioning information and the feature map.

7. The method according to claim 6, characterized in that, After determining the robot target position based on the robot satellite positioning data, the robot motion data, the robot positioning information, the binocular environmental image, the target environmental feature information, and the feature map, the method further includes: Based on the feature map and map boundaries, the robot's travel path is set; The robot's movement is controlled based on the robot's travel path.

8. A robot positioning device, characterized in that, The device includes: The data fusion module is used to fuse robot satellite positioning data and robot motion data to obtain robot positioning information; The feature detection module is used to perform feature detection on the binocular environment image to obtain target environment feature information; The map generation module is used to generate a feature map based on the robot's positioning information and the target environment feature information; The location determination module is used to determine the target location of the robot based on the robot's satellite positioning data, the robot's motion data, the robot's positioning information, the binocular environmental image, the target environmental feature information, and the feature map.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images