Positioning method, device, electronic device and storage medium

Abstract

The present invention discloses a positioning method, device, electronic device and storage medium. The method is applied to an electronic device, the electronic device includes at least two top view sensors, and the method includes: acquiring sensor data, the sensor data includes horizontal view environment data and top view environment data; processing the sensor data; generating a horizontal view grid map and a top view grid map according to the processed sensor data; and positioning the electronic device according to the processed sensor data, the top view grid map and the horizontal view grid map. By using this method, the robustness of positioning is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of robotics technology, and in particular to a positioning method, device, electronic device, and storage medium. Background Art

[0002] With the development of science and technology, electronic devices have been widely used. There are many types of electronic devices. Take robots as an example. Robots are the common name for automatic control machines. Automatic control machines include all machines that simulate human behavior or thoughts and simulate other organisms. At present, the more common robots are mobile robots and fixed robots (such as robotic arms).

[0003] Among them, it is essential to locate the mobile robot during its application. At present, the personnel mobility in the robot's working scene is large and the environment is complex, so the robot is easily affected by environmental factors during the working process, resulting in positioning failure.

[0004] Therefore, how to improve the robustness of electronic equipment positioning is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present invention provide a positioning method, an apparatus, an electronic device and a storage medium. The method is used to perform positioning through a top-view grid map and a head-view grid map, thereby effectively improving the robustness of positioning of the electronic device.

[0006] In a first aspect, an embodiment of the present invention provides a positioning method, which is applied to an electronic device, wherein the electronic device includes at least two top view sensors, and the method includes:

[0007] Acquiring sensor data, wherein the sensor data includes horizontal view environment data and top view environment data;

[0008] processing the sensor data;

[0009] Generate a head-on view grid map and a top-view grid map based on the processed sensor data;

[0010] The electronic device is positioned according to the processed sensor data, the top view grid map and the head view grid map.

[0011] This embodiment combines the top-view grid map and the head-view grid map for positioning, which effectively improves the robustness of the electronic device positioning.

[0012] Optionally, the processing the sensor data includes:

[0013] preprocessing the sensor data;

[0014] Convert the preprocessed sensor data to the body coordinate system;

[0015] Optimize sensor data after coordinate system conversion;

[0016] The processed top view environment data and the processed head view environment data are obtained according to the sensor data.

[0017] This embodiment refines the specific operation of processing sensor data to ensure that the processed sensor data can achieve positioning. Optionally, the preprocessing of the sensor data includes:

[0018] aligning the at least one sensor data based on a timestamp;

[0019] Stitching the aligned top view environment data;

[0020] Extract point cloud information of surface edges in the top view environment data.

[0021] This embodiment refines the technical means for preprocessing sensor data, effectively associates sensor data based on time, and effectively extracts point cloud information of surface edges in top-view environment data for generating a top-view grid map.

[0022] Optionally, the sensor data after optimizing the coordinate system transformation includes:

[0023] The sensor data after the coordinate system conversion is processed by brute force matching.

[0024] This embodiment refines the specific technical means of optimizing the sensor data after the coordinate system is transformed, so that the top view grid map and the head view grid map generated based on the sensor data after the coordinate system is optimized and transformed can be positioned more accurately.

[0025] Optionally, positioning the electronic device according to the processed sensor data, the top view grid map and the horizontal view grid map includes:

[0026] Performing closed-loop detection on the head-up grid map to obtain a head-up matching rate;

[0027] Performing closed-loop detection on the top-view grid map to obtain a top-view matching rate;

[0028] When the head-on view matching rate and / or the top-view matching rate is greater than a set threshold, the global position and posture of the electronic device is determined.

[0029] This embodiment refines the specific technical means of positioning, and determines the global posture of the electronic device based on the head-on matching rate and the top-on matching rate, which can ensure that the positioning result is more accurate.

[0030] Optionally, a head-on view grid map and a top-on view grid map are generated based on the processed sensor data, including:

[0031] Generate a head-up raster map based on the processed head-up environment data;

[0032] Generate a top-view raster map based on the processed top-view environment data.

[0033] This embodiment refines the technical means for generating a head-on view grid map and a top-view grid map. This embodiment can effectively combine the head-on view and top-view environments of the electronic device for precise positioning.

[0034] In a second aspect, an embodiment of the present invention further provides a positioning device, the device comprising:

[0035] An acquisition module, used for acquiring sensor data, wherein the sensor data includes horizontal environment data and top environment data;

[0036] A processing module, used for processing the sensor data;

[0037] A generation module, used for generating a horizontal view grid map and a top view grid map according to the processed sensor data;

[0038] A positioning module is used to position the electronic device according to the processed sensor data, the top view grid map and the head view grid map.

[0039] The device combines the top-view grid map and the head-view grid map for positioning, effectively improving the robustness of electronic equipment positioning.

[0040] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0041] one or more processors;

[0042] A storage device for storing one or more programs;

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement the positioning method provided by the embodiment of the present invention.

[0044] The electronic device in this embodiment effectively improves the robustness of positioning.

[0045] Optionally, the electronic device further includes: a top view sensor and a head-up view sensor;

[0046] The head-up sensor is used to collect head-up environment data in the running direction of the electronic device;

[0047] The top view sensor is used to collect top view environment data above the electronic device. This embodiment further includes a top view sensor and a head-up view sensor, and the positioning accuracy is effectively improved based on the data collected by the top view sensor and the head-up view sensor.

[0048] Optionally, the number of the top view sensors is at least two, each of the top view sensors has a different angle with the horizontal direction, and a common viewing area of ​​a set ratio exists in the field of view angles between two adjacent top view sensors.

[0049] This embodiment limits the number and position of each top view sensor, and limits the field of view angle between adjacent top view sensors to a common view area of ​​a set ratio, thereby ensuring the efficiency of stitching and further improving the positioning accuracy.

[0050] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the positioning method provided by an embodiment of the present invention, thereby effectively improving the robustness of positioning.

[0051] The embodiment of the present invention provides a positioning method, device, electronic device and storage medium, firstly obtaining sensor data, wherein the sensor data includes horizontal environment data and top environment data; then processing the sensor data; secondly generating a horizontal grid map and a top grid map according to the processed sensor data; finally positioning the electronic device according to the processed sensor data, the top grid map and the horizontal grid map. By using the above technical solution, the influence of the environment on the positioning of the electronic device is avoided by combining the top grid map and the horizontal grid map, thereby improving the robustness of positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a flow chart of a positioning method provided in Embodiment 1 of the present invention;

[0053] Figure 1a A schematic diagram of the structure of an indoor robot provided by the present invention;

[0054] Figure 1b A schematic diagram of the structure of another indoor robot provided by the present invention;

[0055] Figure 1c A schematic diagram of a flow chart of another positioning method provided by the present invention;

[0056] Figure 1d A schematic diagram of a flow chart of another positioning method provided by the present invention;

[0057] Figure 1e A schematic diagram of a process of a multi-layer grid map construction and positioning method provided by the present invention;

[0058] Figure 2 A schematic diagram of the structure of a positioning device provided in Embodiment 2 of the present invention;

[0059] Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention;

[0060] Figure 4 A schematic diagram of the structure of a storage medium provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0062] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0063] The term “including” and its variations used in the present invention are open inclusions, that is, “including but not limited to.” The term “based on” means “based at least in part on.” The term “one embodiment” means “at least one embodiment.”

[0064] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0065] Embodiment 1

[0066] Figure 1This is a flow chart of a positioning method provided in Embodiment 1 of the present invention. The method can be applicable to indoor positioning of electronic devices. The method can be performed by a positioning device, wherein the device can be implemented by software and / or hardware and is generally integrated on the electronic device. Among them, the electronic device can be a device that can move actively or passively. Exemplarily, the electronic device can be a robot. The application scenario of the robot is not limited and can be indoors or outdoors. It should be noted that this embodiment describes the electronic device by taking an indoor robot as an example, and the electronic device is not limited here. The implementation method of electronic devices other than indoor robots is the same or similar to the implementation method of indoor robots.

[0067] The robots in the present invention include indoor robots and outdoor robots. Outdoor robots can be considered as robots that can move outdoors. Indoor robots can be considered as robots that can move indoors. Since the working scenes of indoor robots are generally in high-dynamic environments such as shopping malls, garages, and supermarkets, the indoor robot's head-on laser radar (especially two-dimensional laser radar) is often blocked, resulting in the inclusion of many dynamic objects in the laser radar data or the laser radar data is completely blocked and invalid. At the same time, the scenes of indoor robots often change, and the traditional solutions of head-on laser mapping and positioning are not very robust.

[0068] The electronic device in the present invention includes at least two top view sensors for collecting top view environment data. The electronic device structure is described by taking an indoor robot as an example: Figure 1a This is a schematic diagram of the structure of an indoor robot provided by the present invention. Figure 1b A schematic diagram of the structure of another indoor robot provided by the present invention is shown in FIG. Figure 1a and Figure 1b , top view sensor, such as dual depth cameras, that is, two depth cameras are located on the top of the indoor robot, which are used to collect top view environmental data of different areas on the top of the indoor robot. The sensors included in the indoor robot include at least: a depth camera, a laser radar and a wheeled odometer, wherein the depth camera is installed facing upwards, and is used to collect top view environmental data. The angle between the depth camera and the horizontal direction is not limited here, as long as there is a common view area between the field of view of two adjacent depth cameras, such as the angle between the depth camera and the horizontal direction is 90°±20°, and at the same time, it is ensured that the two cameras have a 1 / 3 common view area, which is convenient for camera calibration and image stitching. The installation of the laser radar, that is, the head-up sensor is not limited, as long as it can collect head-up environmental data. Exemplarily, the laser radar is installed in a horizontal and forward direction, and the positive direction is 0° with the horizontal direction.

[0069] When the robot is working indoors, the wheel odometer, lidar, and top-view depth camera are first calibrated. Then, the horizontal environment and top-view environment maps are established, namely, the horizontal grid map and the top-view grid map. After the map is established, the indoor robot can be positioned through the horizontal grid map and the top-view grid map. Specifically, Figure 1 As shown, a positioning method provided in Embodiment 1 of the present invention includes the following steps:

[0070] S110 , acquiring sensor data, where the sensor data includes horizontal view environment data and top view environment data.

[0071] In this embodiment, the sensor data can be considered as data collected by sensors located on the electronic device. The specific content of the sensor data is not limited here, and can be determined according to the type of sensor included in the electronic device. For example, the sensor data may include: environmental data collected by a top-view sensor (i.e., top-view environmental data), environmental data collected by a head-up sensor (i.e., head-up environmental data), and data collected by a wheel odometer, etc. The environmental data includes head-up environmental data and top-view environmental data, and the environmental data can be considered as data collected by sensors that characterize the surrounding environment of the electronic device.

[0072] Head-up environmental data can be considered as environmental data collected by head-up sensors in the direction of the electronic device, such as lidar. Top-view environmental data can be considered as environmental data collected by top-view sensors on the top of the electronic device, such as depth cameras.

[0073] Top-view environment data, such as ceiling information, is relatively fixed and has a low probability of being blocked. Therefore, top-view environment data can maintain stable positioning when the horizontal-view environment data is blocked.

[0074] Before achieving positioning, this step may first obtain sensor data collected by sensors on the electronic device so as to process the sensor data for positioning, and the acquisition method is not limited here. For example, the processor of the electronic device communicates with the sensor to obtain the sensor data collected by the sensor.

[0075] S120: Process the sensor data.

[0076] After acquiring the sensor data, this step can process the sensor data so as to establish a grid map. A grid map is a map expression method that divides a spatial plane into grids of a certain resolution, and the value in the grid is the probability that the current position is occupied. In the present invention, the grid map includes a top view grid map and a horizontal view grid map.

[0077] There is no limitation on how to process the sensor data here, and different sensor data corresponds to different processing methods.

[0078] In one embodiment, since the top view sensor data is collected by at least two top view sensors, processing the top view sensor data may include an operation of splicing the top view sensor data collected by at least two top view sensors to better perform positioning.

[0079] In addition, in order to improve positioning efficiency, when processing top-view environment data, point cloud information of surface edges in the top-view environment data can be extracted; point cloud information of surface edges in the horizontal environment data can also be extracted. For accurate positioning, when processing sensor data, at least one sensor data can be time-stamp aligned.

[0080] S130 , generating a horizontal view grid map and a top view grid map according to the processed sensor data.

[0081] In the present invention, the head-up grid map can be considered as a map established based on the environment in the head-up direction of the electronic device, and the top-view grid map can be considered as a map established based on the environment in the top-view direction of the electronic device. Exemplarily, the head-up grid map can be a grid map generated based on head-up environment data. The top-view grid map can be a grid map generated based on top-view environment data. The data used when establishing the head-up grid map and the top-view grid map are not limited here.

[0082] When generating a grid map, an empty grid map is first generated. After the point cloud information (i.e., the processed top view environment data and the processed head-on view environment data in the processed sensor data) is obtained, the point cloud information can be mapped to the grid map to update the grid map, and the corresponding grid probability can be calculated.

[0083] When the top-view environment of the electronic device is single, this embodiment can combine the head-up grid map to eliminate repeated top-view environment data to avoid the problem of low positioning efficiency caused by the single top-view environment. When there is a large flow of people in the head-up environment or the head-up environment changes dynamically, this embodiment can combine the head-up grid map to improve positioning accuracy.

[0084] S140: Positioning the electronic device according to the processed sensor data, the top view grid map, and the horizontal view grid map.

[0085] After the top view grid map and the head-up view grid map are generated, this step may perform positioning based on the processed sensor data, the top view grid map, and the head-up view grid map.

[0086] In one embodiment, this embodiment can perform posture prediction based on the wheel odometer data in the processed sensor data, and then implement global posture determination based on the top view grid map and the head-up view grid map.

[0087] When positioning is performed based on the top-view grid map and the head-up grid map, closed-loop detection can be first performed on the head-up grid map and the top-view grid map, and then the global pose is optimized and output based on the global pose graph.

[0088] For example, closed loop detection is performed by calculating the matching rate between the point cloud information corresponding to the sensor data and the grid map. The specific formula is as follows:

[0089]

[0090]

[0091] Where: k: represents the number of pre-processed laser points in the current frame; T: represents the position of the current frame in the map; h i : The laser point in the current frame after the laser point has been transformed by posture T; μ, σ: Gaussian distribution parameters representing the height value of a laser point in the map within the grid.

[0092] Among them, Score represents the matching situation between the current point cloud information and the grid map, that is, the matching rate. Score is in the range of 0-1. The larger the score, the better the match and the more likely it is a closed loop. After determining the score, the relative position of the poses of the two nodes can be obtained, which can then be added to the pose graph optimization.

[0093] A node is an abstract concept, representing the information encapsulated in a measurement, including point cloud information and pose information. The relative pose relationship between two nodes obtained by the closed-loop information can be added to the optimization. After optimization, the pose information of the nodes will be adjusted, and the optimized global pose result will be output eventually.

[0094] When optimizing the global pose graph, the cost function is as follows:

[0095]

[0096]

[0097] The function h is as follows, calculating the relative position between two poses:

[0098]

[0099] Where: c i ,c j : The pose information of nodes i, j; R i : The rotation vector of node i; t i ,tj : The position translation vector of nodes i, j; θ i ,θ j : The pose angle vector of nodes i and j; Z ij : Laser matching observation χ2 between nodes i and j: pose graph residual.

[0100] A positioning method provided in a first embodiment of the present invention first acquires sensor data, the sensor data including horizontal environment data and top environment data; then processes the sensor data; secondly generates a horizontal grid map and a top grid map according to the processed sensor data; and finally positions the electronic device according to the processed sensor data, the top grid map and the horizontal grid map. By using the above technical solution, the influence of the environment on the positioning of the electronic device is avoided by combining the top grid map and the horizontal grid map, thereby improving the robustness of positioning.

[0101] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0102] In one embodiment, processing the sensor data includes:

[0103] preprocessing the sensor data;

[0104] Convert the preprocessed sensor data to the body coordinate system;

[0105] Optimize sensor data after coordinate system conversion;

[0106] The processed top view environment data and the processed head view environment data are obtained according to the sensor data.

[0107] When processing sensor data, the sensor data may be preprocessed first. The preprocessing means are not limited and may be determined based on the specific content of the sensor data. For example, the corresponding relationship between sensor data may be established based on time; the top view environment data may be spliced; and point cloud information that is easy to locate in the top view environment data may be extracted, and point cloud information in the horizontal view environment data may be extracted.

[0108] Exemplarily, the preprocessing means of the head-up sensor data in the sensor data include, but are not limited to: timestamp alignment, extracting features in the aligned head-up sensor data, and segmenting point cloud information of surface edges in the head-up sensor data.

[0109] In order to achieve positioning, the pre-processed sensor data can be converted to the body coordinate system to locate the electronic device in the body coordinate system. The conversion means are not limited here.

[0110] After the coordinate system conversion, the sensor data after the coordinate system conversion can be optimized for better positioning. The optimization method is not limited, such as iterative closest point (ICP), ICP variants and brute force matching.

[0111] After optimizing the sensor data after the coordinate system is converted, the processed top view environment data and the processed head-on view environment data can be obtained according to the sensor data, such as reading the optimized top view environment data in the sensor data as the processed top view environment data, and reading the optimized head-on view environment data in the sensor data as the processed head-on view environment data.

[0112] In one embodiment, the sensor data after the coordinate system is optimized includes:

[0113] The sensor data after the coordinate system conversion is processed by brute force matching.

[0114] By processing the sensor data after the coordinate system is converted through brute force matching, the influence of initial value sensitivity can be eliminated. The means of brute force matching are not limited here, such as optimizing the converted data through the Correlation Scan Match (CSM) algorithm. CSM can calculate the relative position between the laser and the map. By optimizing the sensor data after the coordinate system is converted through CSM, the positioning result can be more accurate.

[0115] In one embodiment, the preprocessing of the sensor data includes:

[0116] aligning the at least one sensor data based on a timestamp;

[0117] Stitching the aligned top view environment data;

[0118] Extract point cloud information of surface edges in the top view environment data.

[0119] This embodiment refines the specific technical means of preprocessing sensors. When preprocessing sensor data, at least one sensor data can be aligned based on the timestamp. The sensor data to be aligned is not limited here and can be determined based on actual needs. For example, the top view environment data and the horizontal view environment data are aligned based on the timestamp.

[0120] Since the top view environment data is collected by at least two top view sensors, the aligned top view environment data may be spliced ​​when preprocessing the sensor data, so as to perform positioning based on the spliced ​​top view environment data. The specific means of splicing are not limited here.

[0121] After splicing the aligned top-view environment data, in order to facilitate the generation of a top-view grid map, this embodiment can extract point cloud information of the edge of the surface in the top-view environment data, so as to map the point cloud information to the top-view grid map to achieve mapping and positioning; in order to facilitate the generation of a head-up grid map, the point cloud information of the edge of the surface in the aligned head-up environment data can be extracted, so as to map the point cloud information to the head-up grid map to achieve mapping and positioning.

[0122] Figure 1c A flow chart of another positioning method provided by the present invention is shown in FIG. Figure 1c , the method comprises the following steps:

[0123] S210: Acquire sensor data, where the sensor data includes horizontal view environment data and top view environment data.

[0124] S220: Align the at least one sensor data based on the timestamp.

[0125] Aligning the at least one sensor data based on the timestamp can be considered as establishing a corresponding relationship between the at least one sensor data based on time, so as to process the aligned top view environment data and further achieve positioning.

[0126] S230, stitching the aligned top view environment data.

[0127] The specific means of stitching the aligned top view environment data is not limited, for example, stitching can be achieved based on the top view environment data in the common view area to stitch the top view environment data collected by each top view sensor. When there is no common view area, the aligned top view environment data can be stitched based on the position of each top view sensor in the electronic device.

[0128] S240: Extracting point cloud information of surface edges in the top-view environment data.

[0129] After the aligned top view environment data is spliced, the present embodiment can extract the point cloud information of the surface edge in the spliced ​​top view environment data for positioning. The specific means of extracting the point cloud information of the surface edge is not limited here.

[0130] S250: convert the pre-processed sensor data into the body coordinate system.

[0131] This example only shows the process of processing the top view environment data, and the specific technical means for pre-processing the remaining data in the sensor data are not limited here.

[0132] S260, optimizing the sensor data after the coordinate system is converted, and obtaining processed top view environment data and processed head view environment data according to the sensor data.

[0133] S270 , generating a horizontal view grid map and a top view grid map according to the processed sensor data.

[0134] S280: Positioning the electronic device according to the processed sensor data, the top view grid map, and the horizontal view grid map.

[0135] In one embodiment, positioning the electronic device according to the processed sensor data, the top view grid map and the head view grid map includes:

[0136] Performing closed-loop detection on the head-up grid map to obtain a head-up matching rate;

[0137] Performing closed-loop detection on the top-view grid map to obtain a top-view matching rate;

[0138] When the head-on view matching rate and / or the top-view matching rate is greater than a set threshold, the global position and posture of the electronic device is determined.

[0139] Figure 1d A flow chart of another positioning method provided by the present invention is shown in FIG. Figure 1d , the method comprises the following steps:

[0140] S310, acquiring sensor data, the sensor data including horizontal environment data and top environment data. S320, processing the sensor data.

[0141] S330 , generating a horizontal view grid map and a top view grid map according to the processed sensor data.

[0142] S340: Perform closed-loop detection on the head-up grid map to obtain a head-up matching rate.

[0143] The head-up matching rate can be considered as the probability of determining the matching situation between the head-up grid map and the head-up environment data. The technical means of closed-loop detection are not limited here, as long as the head-up matching rate can be determined.

[0144] The present invention can perform closed-loop detection on the head-up grid map based on the head-up grid map and the head-up environment data to obtain the head-up matching rate.

[0145] S350: Perform closed-loop detection on the top-view grid map to obtain a top-view matching rate.

[0146] The top view matching rate can be considered as the probability of determining the matching situation between the top view grid map and the top view environment data. The technical means for performing closed loop detection are not limited here, as long as the top view matching rate can be determined.

[0147] The present invention can perform closed-loop detection on the top-view grid map based on the top-view grid map and the top-view environment data to obtain the top-view matching rate.

[0148] When determining the head-on view matching rate and the top view matching rate, closed loop detection, also known as loop closure detection, can be performed on the head-on view grid map and the top view grid map respectively to determine the corresponding top view matching rate and head-on view matching rate. The execution order of determining the head-on view matching rate and the top view matching rate is not limited here, and they can be determined in parallel or in sequence.

[0149] S360: When the horizontal view matching rate and / or the top view matching rate is greater than a set threshold, determine the global position and posture of the electronic device.

[0150] When the head-on view matching rate and / or the top view matching rate is greater than a set threshold, the present embodiment can determine the global posture of the electronic device based on the top view grid map and the head-on view grid map. For example, the posture graph residual is determined based on a posture graph optimization technique to determine the global posture. The set threshold is not limited. The head-on view matching rate and the top view matching rate can correspond to different set thresholds.

[0151] In one embodiment, generating a head-on view grid map and a top-on view grid map based on processed sensor data includes:

[0152] Generate a head-up raster map based on the processed head-up environment data;

[0153] Generate a top-view raster map based on the processed top-view environment data.

[0154] In this embodiment, the head-up grid map is generated based on the head-up environment data, and the top-down grid map is generated based on the top-down environment data.

[0155] It should be noted that in the initial stage of map construction, an empty top view grid map and an empty head-on view grid map can be first established. After obtaining the sensor data, the head-on view grid map can be updated based on the processed head-on view environment data, and the top view grid map can be updated based on the processed top view environment data.

[0156] This embodiment builds a map by fusing top-view environment data and head-up environment data, that is, mapping and positioning are performed based on the fusion of top-view sensors and head-up sensors. This solution comprehensively considers the measurement data characteristics of the head-up lidar (i.e., head-up sensor) and the top-view depth camera (i.e., top-view sensor) and the application scenario characteristics, gives full play to the advantages of each sensor data, and realizes accurate real-time mapping and highly robust positioning.

[0157] Figure 1e A schematic diagram of a multi-layer grid map construction and positioning method provided by the present invention. Figure 1e Taking the number of top view sensors as two as an example, the method includes:

[0158] S1. Obtain data from each sensor.

[0159] The sensor data obtained include lidar, wheel odometer and depth camera data, namely top view environment data and head-up environment data.

[0160] S2. Data preprocessing.

[0161] S2.1. Align the timestamps of the sensor data and splice the dual top-view depth camera data, i.e., the top-view environment data collected by the two top-view sensors;

[0162] S2.2. Feature extraction of top view environment data to segment the point cloud information of surface edges.

[0163] S3. Pose prediction using wheel odometry.

[0164] S4. Coordinate system conversion.

[0165] The point cloud information of the laser radar and the surface edge point cloud information of the top view environment data are transferred to the body coordinate system to obtain the converted point cloud information, wherein the body coordinate system can be customized or the coordinate system of the wheel odometer.

[0166] S5. Optimize point cloud information.

[0167] The converted point cloud information is used to optimize the current posture through the CSM algorithm, where the optimized posture can be considered as the position of the current body coordinate system in the grid map. The current point cloud information is in the body coordinate system, and the posture of the body coordinate system in the map can be corrected by matching the current point cloud information and the map.

[0168] S6. Update the probability occupancy grid map.

[0169] S6.1. Update the head-up raster map with head-up laser data;

[0170] S6.2. The top view surface edge point cloud updates the top view grid map.

[0171] S7. Closed loop detection.

[0172] S7.1. Detect head-up closed loops using head-up laser data and head-up grid maps;

[0173] S7.2. Detect top view closed loops through top view surface edge point clouds and top view grid maps;

[0174] S8. Optimize the global pose graph and output the global pose.

[0175] Among them, the head-up laser data can be considered as the processed head-up environment data, and the top-view surface edge point cloud can be considered as the processed top-view environment data.

[0176] Embodiment 2

[0177] Figure 2 This is a schematic diagram of the structure of a positioning device provided in Embodiment 2 of the present invention. The device can be applicable to the situation of positioning an electronic device, wherein the device can be implemented by software and / or hardware and is generally integrated on the electronic device.

[0178] like Figure 2 As shown, the device comprises:

[0179] An acquisition module 21 is used to acquire sensor data, wherein the sensor data includes horizontal environment data and top environment data;

[0180] A processing module 22, configured to process the sensor data;

[0181] A generating module 23, used for generating a horizontal view grid map and a top view grid map according to the processed sensor data;

[0182] The positioning module 24 is used to position the electronic device according to the processed sensor data, the top view grid map and the head view grid map.

[0183] In this embodiment, the device first acquires sensor data through an acquisition module 21, and the sensor data includes head-on environment data and top-view environment data; secondly, the sensor data is processed through a processing module 22; then, a head-on grid map and a top-view grid map are generated according to the processed sensor data through a generation module 23; finally, the electronic device is positioned according to the processed sensor data, the top-view grid map and the head-on grid map through a positioning module 24.

[0184] This embodiment provides a positioning device, which combines a top-view grid map and a head-view grid map to avoid the influence of the environment on the positioning of the electronic device, thereby improving the robustness of the positioning.

[0185] In one embodiment, the processing module 22 includes:

[0186] A preprocessing unit 221, used for preprocessing the sensor data;

[0187] A conversion unit 222, used to convert the pre-processed sensor data into a body coordinate system;

[0188] The optimization unit 223 is used to optimize the sensor data after the coordinate system is converted, and obtain the processed top view environment data and the processed head view environment data according to the sensor data.

[0189] In one embodiment, the preprocessing unit 221 is specifically used for:

[0190] aligning the at least one sensor data based on a timestamp;

[0191] Stitching the aligned top view environment data;

[0192] Extract point cloud information of surface edges in the top view environment data.

[0193] In one embodiment, the optimization unit 223 is specifically used for:

[0194] The sensor data after the coordinate system conversion is processed by brute force matching.

[0195] In one embodiment, the positioning module 24 is specifically used for:

[0196] Performing closed-loop detection on the head-up grid map to obtain a head-up matching rate;

[0197] Performing closed-loop detection on the top-view grid map to obtain a top-view matching rate;

[0198] When the head-on view matching rate and / or the top-view matching rate is greater than a set threshold, the global position and posture of the electronic device is determined.

[0199] In one embodiment, the generating module 23 is specifically used for:

[0200] Generate a head-up raster map based on the processed head-up environment data;

[0201] Generate a top-view raster map based on the processed top-view environment data.

[0202] The above positioning device can execute the positioning method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0203] Embodiment 3

[0204] Figure 3 This is a schematic diagram of the structure of an electronic device provided by Embodiment 3 of the present invention. Figure 3 As shown, the electronic device provided by the third embodiment of the present invention includes: one or more processors 31 and a storage device 32; the processor 31 in the electronic device can be one or more, Figure 3 A processor 31 is taken as an example; the storage device 32 is used to store one or more programs; the one or more programs are executed by the one or more processors 31, so that the one or more processors 31 implement the positioning method as described in any one of the embodiments of the present invention.

[0205] The electronic device may further include: an input device 33 and an output device 34 .

[0206] In one embodiment, the electronic device further comprises: a top view sensor and a head-up view sensor;

[0207] The head-up sensor is used to collect head-up environment data in the running direction of the electronic device;

[0208] The top view sensor is used to collect top view environment data above the electronic device.

[0209] In one embodiment, the number of the top view sensors is at least two, each of the top view sensors has a different angle with the horizontal direction, and a common viewing area of ​​a set ratio exists between the field of view angles of two adjacent top view sensors.

[0210] The top view sensor in the present invention can collect top view environment data in the same area to improve the collection accuracy. In this embodiment, each top view sensor can collect top view environment data in different areas above the electronic device to improve the positioning accuracy by expanding the collection range.

[0211] In the present invention, each top view sensor may not have a common view area to collect more top view environment data. In this embodiment, the field of view angles between adjacent top view sensors may have a set ratio of common view areas to effectively splice the top view environment data. The set ratio is not limited, such as one third or one quarter.

[0212] The processor 31, storage device 32, input device 33 and output device 34 in the electronic device may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0213] The storage device 32 in the electronic device is a computer-readable storage medium, which can be used to store one or more programs, and the program can be a software program, a computer executable program, and a module, such as the program instructions / modules corresponding to the positioning method provided in the first embodiment of the present invention (for example, the attached Figure 2 The modules in the positioning device shown include: an acquisition module 21, a processing module 22, a generation module 23 and a positioning module 24). The processor 31 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the storage device 32, that is, the positioning method in the above method embodiment is implemented.

[0214] The storage device 32 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the storage device 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the storage device 32 may further include a memory remotely arranged relative to the processor 31, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0215] The input device 33 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 34 may include a display device such as a display screen.

[0216] Furthermore, when one or more programs included in the electronic device are executed by the one or more processors 31, the program performs the following operations:

[0217] Acquiring sensor data, wherein the sensor data includes horizontal view environment data and top view environment data;

[0218] processing the sensor data;

[0219] Generate a head-on view grid map and a top-view grid map based on the processed sensor data;

[0220] The electronic device is positioned according to the processed sensor data, the top view grid map and the head view grid map.

[0221] Embodiment 4

[0222] Embodiment 4 of the present invention provides a computer-readable storage medium, Figure 4 A schematic diagram of a storage medium provided in Embodiment 4 of the present invention is shown in FIG. Figure 4 As shown, a computer program 410 is stored thereon, and when the computer program 410 is executed by the processor 42, it is used to perform a positioning method, the method comprising: acquiring sensor data, the sensor data comprising horizontal view environment data and top view environment data;

[0223] processing the sensor data;

[0224] Generate a head-on view grid map and a top-view grid map based on the processed sensor data;

[0225] The electronic device is positioned according to the processed sensor data, the top view grid map and the head view grid map.

[0226] Optionally, when the computer program 410 is executed by the processor 42, it can also be used to execute the positioning method provided by any embodiment of the present invention.

[0227] The computer-readable storage medium 41 of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable storage medium 41 may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of the computer-readable storage medium 41 include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination thereof. The computer-readable storage medium 41 may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0228] Computer readable storage medium 41 may include a data signal propagated in baseband or as part of a carrier wave, wherein a computer readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable storage medium 41 may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0229] The program code contained in the computer-readable storage medium 41 may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination thereof.

[0230] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0231] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A positioning method, characterized in that: Applied to an electronic device, the electronic device includes at least two top view sensors, the two top view sensors are at least two adjacent depth cameras, used to collect top view environment data above the electronic device, and the field of view angles of the at least two depth cameras have a common view area, the method includes: Acquiring sensor data, wherein the sensor data includes horizontal view environment data and top view environment data; The head-up environmental data is environmental data of the running direction of the electronic device collected by the head-up sensor, and the top-view environmental data is environmental data of the top of the electronic device collected by the top-view sensor; processing the sensor data; Generate a head-on view grid map and a top-view grid map based on the processed sensor data; Positioning the electronic device according to the processed sensor data, the top view grid map and the head view grid map; Positioning the electronic device according to the processed sensor data, the top view grid map and the head view grid map includes: Performing closed-loop detection on the head-up grid map to obtain a head-up matching rate; Performing closed-loop detection on the top-view grid map to obtain a top-view matching rate; When the head-on view matching rate and / or the top-on view matching rate is greater than a set threshold, determining the global position and posture of the electronic device; The horizontal grid map is a map established based on the environment in the horizontal direction of the electronic device, and the top grid map is a map established based on the environment in the top direction of the electronic device; The head-on view matching rate is the probability of the head-on view grid map matching the head-on view environment data, and the top view matching rate is the probability of the top view grid map matching the top view environment data; The closed loop detection is calculated by the matching rate between the point cloud information corresponding to the sensor data and the grid map. The specific calculation formula is as follows: Among them, Score is the matching rate between the current point cloud information and the grid map, the value of the matching rate is in the range of 0-1, and the matching rate is proportional to the closed-loop probability; k is the number of pre-processed laser points in the current frame; T is the position of the current frame in the map; h i is the laser point in the current frame after the laser point has been transformed through the pose T; μ, σ: Gaussian distribution parameters representing the height value of the laser point in the map falling within the grid; After determining the matching rate of the current point cloud information and the grid map, the pose graph is optimized according to the relative positions of the poses of the two nodes; the nodes are used to characterize the information encapsulated in a single measurement, including point cloud information and pose information; the pose graph is optimized according to the relative pose relationship between the two nodes, the pose information of the nodes is adjusted after optimization, and the optimized global pose result is output, wherein the cost function for optimizing the global pose graph is as follows: The function h is as follows, which is used to calculate the relative position between two poses: Where: c i ,c j : The pose information of nodes i, j; R i : The rotation vector of node i; t i ,t j : The position translation vector of nodes i, j; θ i ,θ j : The pose angle vector of nodes i and j; Z ij : Laser matching observation χ2 between nodes i and j: pose graph residual.

2. The method according to claim 1, characterized in that The processing of the sensor data comprises: preprocessing the sensor data; Convert the preprocessed sensor data to the body coordinate system; Optimize sensor data after coordinate system conversion; The processed top view environment data and the processed head view environment data are obtained according to the sensor data.

3. The method according to claim 2, characterized in that The preprocessing of the sensor data comprises: aligning the at least one sensor data based on a timestamp; Stitching the aligned top view environment data; Extract point cloud information of surface edges in the top view environment data.

4. The method according to claim 2, characterized in that: The sensor data after the coordinate system is optimized and transformed includes: The sensor data after the coordinate system conversion is processed by brute force matching.

5. The method according to claim 1, characterized in that Generate a head-on view grid map and a top-view grid map based on the processed sensor data, including: Generate a head-up raster map based on the processed head-up environment data; Generate a top-view raster map based on the processed top-view environment data.

6. A positioning device, characterized in that: The device comprises: An acquisition module, used for acquiring sensor data, wherein the sensor data includes horizontal environment data and top environment data; The head-up environmental data is environmental data of the running direction of the electronic device collected by the head-up sensor, and the top-view environmental data is environmental data of the top of the electronic device collected by the top-view sensor; A processing module, used for processing the sensor data; A generation module, used for generating a horizontal view grid map and a top view grid map according to the processed sensor data; A positioning module, used for positioning the electronic device according to the processed sensor data, the top view grid map and the head view grid map; Positioning module, specifically used for: Performing closed-loop detection on the head-up grid map to obtain a head-up matching rate; Performing closed-loop detection on the top-view grid map to obtain a top-view matching rate; When the head-on view matching rate and / or the top-on view matching rate is greater than a set threshold, determining the global position and posture of the electronic device; The horizontal grid map is a map established based on the environment in the horizontal direction of the electronic device, and the top grid map is a map established based on the environment in the top direction of the electronic device; The head-on view matching rate is the probability of the head-on view grid map matching the head-on view environment data, and the top view matching rate is the probability of the top view grid map matching the top view environment data; The closed loop detection is calculated by the matching rate between the point cloud information corresponding to the sensor data and the grid map. The specific calculation formula is as follows: Among them, Score is the matching rate between the current point cloud information and the grid map, the value of the matching rate is in the range of 0-1, and the matching rate is proportional to the closed-loop probability; k is the number of pre-processed laser points in the current frame; T is the position of the current frame in the map; h i is the laser point in the current frame after the laser point has been transformed through the pose T; μ, σ: Gaussian distribution parameters representing the height value of the laser point in the map falling within the grid; After determining the matching rate of the current point cloud information and the grid map, the pose graph is optimized according to the relative positions of the poses of the two nodes; the nodes are used to characterize the information encapsulated in a single measurement, including point cloud information and pose information; the pose graph is optimized according to the relative pose relationship between the two nodes, the pose information of the nodes is adjusted after optimization, and the optimized global pose result is output, wherein the cost function for optimizing the global pose graph is as follows: The function h is as follows, which is used to calculate the relative position between two poses: Where: c i ,c j : The pose information of nodes i, j; R i : The rotation vector of node i; t i ,t j : The position translation vector of nodes i, j; θ i ,θ j : The pose angle vector of nodes i and j; Z ij : Laser matching observation χ2 between nodes i and j: pose graph residual.

7. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the positioning method as described in any one of claims 1-5.

8. The electronic device according to claim 7, characterized in that: Also includes: Top view sensor and head-up view sensor; The head-up sensor is used to collect head-up environment data in the running direction of the electronic device; The top view sensor is used to collect top view environment data above the electronic device.

9. The electronic device according to claim 8, characterized in that: The number of the top view sensors is at least two, each of the top view sensors has a different angle with the horizontal direction, and a common viewing area with a set ratio exists between the field of view angles of two adjacent top view sensors.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the positioning method as described in any one of claims 1 to 5 is implemented.

Images