Terminal relocation method and device

Abstract

The present invention discloses a terminal relocation method and device, and relates to the field of intelligent driving technology. A specific implementation of the method includes: in response to the relocation operation of the terminal, obtaining the current frame point cloud data collected by the terminal, performing signature processing on the current frame point cloud data, and obtaining the current frame signature; obtaining the frame signature of each key frame point cloud data in the point cloud map, calculating the difference value between the current frame signature and each frame signature, and determining the first preset number of key frame point cloud data in the order of the difference value from small to large; determining the posture of each key frame point cloud data in the first preset number of key frame point cloud data, calculating the matching value and posture transformation value of each key frame point cloud data and the current frame point cloud data; superimposing the posture of the key frame point cloud data with the largest matching degree on the corresponding posture transformation value, and obtaining the current posture of the terminal in the point cloud map. This implementation can realize the effective selection of relocation key frames and ensure the accuracy of the relocation results.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a terminal relocation method and device. Background Art

[0002] LiDAR is essential for autonomous driving terminals (such as robots and vehicles). The LiDAR market is currently in rapid development. LiDAR can use pulsed laser to measure dynamic distances, create and update three-dimensional maps, and make centimeter-level terminal positioning possible.

[0003] When the terminal fails to locate due to sensor data errors, insufficient environmental information, accumulated errors, etc. during the first initialization or after a long period of operation, relocation is required to ensure normal positioning of the terminal. Relocation requires finding the correct position on the pre-established map based on the current radar data or visual data.

[0004] At present, there are few frame signatures designed based on radar data for relocation problems, such as using the highest point in each grid as a feature point and a two-dimensional vector composed of a two-dimensional grid arranged in space as a frame signature. However, the feature point data used in this method is greatly interfered by mobile terminals, which affects its robustness. In addition, the designed two-dimensional vector signature is not rotationally invariant. When calculating whether two frames of radar data are similar, if a two-dimensional vector is used, it needs to be traversed and aligned column by column before comparison, which requires a large amount of calculation; if the accumulated one-dimensional vector is used for comparison, the accuracy may decrease. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a terminal relocation method and device, which can at least solve the problem of low positioning accuracy caused by using one-dimensional vector comparison when the terminal is relocated in the prior art.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a terminal relocation method is provided, including:

[0007] In response to the relocation operation of the terminal, the current frame point cloud data collected by the terminal is acquired, and the current frame point cloud data is signature processed to obtain the current frame signature;

[0008] Obtaining a frame signature of each key frame point cloud data in the point cloud map, calculating a difference value between the current frame signature and each frame signature, and determining a first preset number of key frame point cloud data in ascending order of the difference value;

[0009] Determine the pose of each key frame point cloud data in the first preset number of key frame point cloud data, and calculate the matching value and pose transformation value of each key frame point cloud data and the current frame point cloud data;

[0010] The pose of the key frame point cloud data with the greatest matching degree is superimposed with the corresponding pose transformation value to obtain the current target pose of the terminal in the point cloud map.

[0011] Optionally, before the responding to the relocation operation of the terminal, the method further includes:

[0012] During the operation of one or more terminals, multiple frames of point cloud data collected by the laser radar are obtained to construct a point cloud map based on the multiple frames of point cloud data;

[0013] According to the distances between adjacent frames of point cloud data in the point cloud map, key frame point cloud data whose distances meet a preset distance range are selected;

[0014] Perform signature processing on each key frame point cloud data to obtain a frame signature of each key frame point cloud data; and use a preset algorithm to calculate the pose of each key frame point cloud data.

[0015] Optionally, performing signature processing on the current frame point cloud data to obtain the current frame signature includes:

[0016] Convert the coordinate value of each point in the Cartesian coordinate system of the current frame point cloud data into a row index, a column index, and a distance between the projection point on the horizontal plane and the origin in a cylindrical coordinate system;

[0017] Arrange all points according to row index and column index to generate two-dimensional data, calculate the inclination of two adjacent points in each column in turn, and mark the two adjacent points whose absolute value of the inclination is greater than or equal to the preset inclination threshold as vertical direction points;

[0018] Calculate the distance variance, horizontal axis coordinate mean and vertical axis coordinate mean of each column of vertical points to determine the relative distance distribution of vertical points between any two columns and the contribution value of each relative distance distribution;

[0019] The contribution values ​​are arranged in ascending order of relative distance distribution to generate an array, and the generated array is determined as a frame signature.

[0020] Optionally, the step of calculating the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of each column of vertical direction points includes:

[0021] Counting the number of vertical points in each column to filter out one or more columns whose number is greater than or equal to a second preset number;

[0022] For each of the one or more columns, the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of the points in the vertical direction are calculated.

[0023] Optionally, determining the relative distance distribution between any two columns includes:

[0024] Determine a third preset number of columns in the order of distance variance from small to large;

[0025] The relative distance distribution between any two columns in the second preset number of columns is calculated according to the distance variance, the horizontal axis coordinate mean and the vertical axis coordinate mean of each column of vertical direction points.

[0026] Optionally, determining a third preset number of columns further includes:

[0027] For a plurality of adjacent columns of data in the third preset number of columns, a column of data having the smallest variance, or a column of data located in the middle of the plurality of columns of data is selected.

[0028] Optionally, arranging the contribution values ​​in ascending order according to the relative distance distribution includes:

[0029] The contribution values ​​of the same relative distance distribution are accumulated, and a statistical chart is constructed based on the relative distance distribution and the accumulated contribution values. The contribution values ​​are arranged in ascending order according to the relative distance distribution in the statistical chart.

[0030] To achieve the above object, according to another aspect of an embodiment of the present invention, a terminal relocation device is provided, comprising:

[0031] A frame signature module, used for obtaining the current frame point cloud data collected by the terminal in response to the relocation operation of the terminal, performing signature processing on the current frame point cloud data, and obtaining the current frame signature;

[0032] A first calculation module is used to obtain a frame signature of each key frame point cloud data in the point cloud map, and calculate a difference value between the current frame signature and each frame signature, so as to determine a first preset number of key frame point cloud data in ascending order of the difference value;

[0033] A second calculation module is used to determine the posture of each key frame point cloud data in the first preset number of key frame point cloud data, and calculate the matching value and posture transformation value of each key frame point cloud data and the current frame point cloud data;

[0034] The posture module is used to superimpose the posture of the key frame point cloud data with the greatest matching degree on the corresponding posture transformation value to obtain the current target posture of the terminal in the point cloud map.

[0035] Optionally, a keyframe module is also included for:

[0036] During the operation of one or more terminals, multiple frames of point cloud data collected by the laser radar are obtained to construct a point cloud map based on the multiple frames of point cloud data;

[0037] According to the distances between adjacent frames of point cloud data in the point cloud map, key frame point cloud data whose distances meet a preset distance range are selected;

[0038] Perform signature processing on each key frame point cloud data to obtain a frame signature of each key frame point cloud data; and use a preset algorithm to calculate the pose of each key frame point cloud data.

[0039] Optionally, the frame signature module is used to:

[0040] Convert the coordinate value of each point in the Cartesian coordinate system of the current frame point cloud data into a row index, a column index, and a distance between the projection point on the horizontal plane and the origin in a cylindrical coordinate system;

[0041] Arrange all points according to row index and column index to generate two-dimensional data, calculate the inclination of two adjacent points in each column in turn, and mark the two adjacent points whose absolute value of the inclination is greater than or equal to the preset inclination threshold as vertical direction points;

[0042] Calculate the distance variance, horizontal axis coordinate mean and vertical axis coordinate mean of each column of vertical points to determine the relative distance distribution of vertical points between any two columns and the contribution value of each relative distance distribution;

[0043] The contribution values ​​are arranged in ascending order of relative distance distribution to generate an array, and the generated array is determined as a frame signature.

[0044] Optionally, the frame signature module is used to:

[0045] Counting the number of vertical points in each column to filter out one or more columns whose number is greater than or equal to a second preset number;

[0046] For each of the one or more columns, the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of the points in the vertical direction are calculated.

[0047] Optionally, the frame signature module is used to:

[0048] Determine a third preset number of columns in the order of distance variance from small to large;

[0049] The relative distance distribution between any two columns in the second preset number of columns is calculated according to the distance variance, the horizontal axis coordinate mean and the vertical axis coordinate mean of each column of vertical direction points.

[0050] Optionally, the frame signature module is used to:

[0051] For a plurality of adjacent columns of data in the third preset number of columns, a column of data having the smallest variance, or a column of data located in the middle of the plurality of columns of data is selected.

[0052] Optionally, the frame signature module is used to:

[0053] The contribution values ​​of the same relative distance distribution are accumulated, and a statistical chart is constructed based on the relative distance distribution and the accumulated contribution values. The contribution values ​​are arranged in ascending order according to the relative distance distribution in the statistical chart.

[0054] To achieve the above objective, according to another aspect of an embodiment of the present invention, a terminal relocation electronic device is provided.

[0055] The electronic device of an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the terminal relocation methods described above.

[0056] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the above terminal relocation methods is implemented.

[0057] According to the solution provided by the present invention, an embodiment of the above invention has the following advantages or beneficial effects: through a pre-constructed point cloud map, the candidate key frames for terminal relocation are simply and effectively determined to screen the target key frames with the highest matching degree, thereby achieving effective selection of relocation key frames and ensuring the accuracy of the relocation results. At the same time, it also reduces the excessive consumption of computing resources and time. In the field of autonomous driving, it can improve driving efficiency, comfort, intelligence and safety.

[0058] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0060] Figure 1 is a schematic diagram of the main flow of a terminal relocation method according to an embodiment of the present invention;

[0061] Figure 2 It is a schematic diagram of the overall implementation process;

[0062] Figure 3 is a flow chart of an optional terminal relocation method according to an embodiment of the present invention;

[0063] Figure 4(a) shows the coordinates of each point in the point cloud data in earth coordinates;

[0064] Figure 4(b) is a schematic diagram of two-dimensional data arrangement;

[0065] FIG4( c ) is a schematic diagram of generating a frame signature through a relative distance histogram;

[0066] Figure 5 It is a schematic diagram of the overall process of frame signature generation;

[0067] Figure 6 is a schematic diagram of main modules of a terminal relocation device according to an embodiment of the present invention;

[0068] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0069] Figure 8 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0070] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0071] It should be noted that the embodiments and features of the embodiments of the present invention can be combined with each other without conflict. The acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations.

[0072] The terms involved in this plan are explained as follows:

[0073] Robustness: The key to the survival of a system in abnormal and dangerous situations. For example, the robustness of a computer software is whether it can survive input errors, disk failures, network overloads, or intentional attacks without shutting down or crashing. This is used to characterize the insensitivity of the control system to characteristic or parameter perturbations.

[0074] SLAM (Simultaneous Localization and Mapping) algorithm: Uses extended Kalman filtering to simultaneously estimate the robot's pose and feature (landmark) positions in state space.

[0075] Frame signature: It can also be called frame feature coding. After the feature extraction of radar frame point cloud data, the extracted features are encoded according to a predetermined strategy, and the resulting code is the frame signature.

[0076] Rotational invariance: When the terminal rotates a certain angle, the frame feature encoding obtained is the same, which is rotational invariance.

[0077] See also Figure 1 , which shows a main flow chart of a terminal relocation method provided by an embodiment of the present invention, includes the following steps:

[0078] S101: in response to a relocation operation on a terminal, obtaining current frame point cloud data collected by the terminal, and performing signature processing on the current frame point cloud data to obtain a current frame signature;

[0079] S102: Obtain a frame signature of each key frame point cloud data in the point cloud map, calculate a difference value between the current frame signature and each frame signature, and determine a first preset number of key frame point cloud data in ascending order of the difference value;

[0080] S103: determining the position and posture of each key frame point cloud data in the first preset number of key frame point cloud data, and calculating the matching value and position and posture transformation value of each key frame point cloud data and the current frame point cloud data;

[0081] S104: Superimposing the corresponding posture transformation value on the posture of the key frame point cloud data with the greatest matching degree to obtain the current target posture of the terminal in the point cloud map.

[0082] In the above implementation, for step S101, this solution mainly uses a multi-line laser radar. Multi-line is relative to single-line and is a product parameter of the laser radar. Specifically:

[0083] 1) Single-line radar has only one laser transmitter and receiver. After the motor rotates, it projects a line to the obstacle in front. It can only detect obstacles of the same height each time and cannot measure the overall contour. It is mostly used in sweeping robots and hotel service robots;

[0084] 2) Currently, there are 4-line, 8-line, 16-line, 32-line, 64-line, and 128-line multi-line radars on the market. There are multiple transmitters and receivers in the vertical direction. Multiple line beams are obtained through the rotation of the motor. The more lines there are, the more perfect the surface contour of the object is, the larger the amount of data to be processed, and the higher the hardware requirements. Multi-line radars are mainly used in unmanned driving scenarios. They can calculate the height information of objects and perform 3D modeling of the surrounding environment.

[0085] This solution pre-establishes the frame signature and pose of the key frame point cloud data. Install the laser radar and processing equipment on one or more terminals and travel along the preset route. The laser radar rotates one circle to obtain a frame of point cloud data; among them, point cloud data refers to a set of vectors in a three-dimensional coordinate system. The laser radar uses an automated method to measure the information of a large number of points on the surface of an object, and then outputs the point cloud data using a certain specific data file, which represents the outer contour of the building around the terminal. The laser radar is connected to the processing equipment, so the processing equipment will continuously obtain multiple frames of point cloud data collected by the laser radar. High-precision point cloud maps are required in autonomous driving scenarios. The point cloud map stores the original scanning point cloud data of the laser radar on the environment. The advantage is that the information is kept intact, and the disadvantage is that the calculation is large, but it cannot be directly used for navigation and obstacle avoidance.

[0086] The point cloud map includes multiple frames of point cloud data. In order to reduce the amount of subsequent calculations, only key frames can be set. A key frame refers to a frame of data obtained by a laser radar scan. Generally, a frame is taken as a key frame at a certain distance (for example, 0.2m), and the middle frame is a non-key frame. Call the frame signature processing program for each key frame to obtain the frame signature of each key frame. Use the SLAM (such as lio-sam) algorithm to process the point cloud data of each key frame to obtain the pose of each key frame, including position and direction. The direction mainly refers to the direction of the front of the vehicle. Store the frame signature and pose of each key frame in the point cloud map in a preset file, such as a txt format file, and parse it when used.

[0087] The point cloud data required for generating the point cloud map can be provided by the terminal being relocated this time, such as terminal A has previously walked on route B, and has collected point cloud data of route B and generated a point cloud map. It can also be provided by one or more other terminals, such as terminal A walking on route B for the first time, but a relocation requirement arises, while terminals C and D have previously walked on route B, and have generated a point cloud map for terminal A to use.

[0088] When the terminal is initialized for the first time in the point cloud map, or after a long period of operation, when positioning fails due to sensor data errors, less environmental information, accumulated errors, etc., repositioning is required to ensure the normal positioning of the terminal. In response to the repositioning operation of the terminal, the laser radar is used to rotate a circle to collect the current frame point cloud data around the terminal, and the frame signature processing program is called to process the current frame point cloud data to obtain the current frame signature.

[0089] Compare the current frame signature with the frame signature of each key frame in the aforementioned preset file for difference or similarity. Assuming that the frame signature consists of 200 sequential numbers, the comparison of two frame signatures is to compare the 200 numbers one by one according to the digital position to see if they are the same. For example, if 0 and 1 in the first position are different, and 1 and 1 in the second position are the same, count how many positions have differences in the numbers, and calculate the difference value (or similarity) to select K (that is, the first preset number, such as 5) frame signatures with the smallest difference value (or maximum similarity).

[0090] For step S103, point cloud registration refers to inputting two point cloud data and outputting a transformation T so that the poses of the two point cloud data overlap as much as possible. The transformation T can be rigid or not. This scheme only considers rigid transformation, that is, the transformation only includes rotation and translation. Point cloud registration can be divided into two steps: coarse registration and fine registration. Coarse registration refers to a relatively rough registration when the transformation between the two point clouds is completely unknown. The main purpose is to provide a better initial value of the transformation for fine registration; the fine registration criterion is to give an initial transformation and further optimize it to obtain a more accurate transformation. The most widely used point cloud fine registration algorithm at present is the iterative closest point (ICP) algorithm.

[0091] The target keyframe cannot be 100% determined by frame signature alone, so further processing is required. The keyframes corresponding to the K frame signatures are used as candidate frames, and the iterative closest point algorithm (ICP) is used to calculate the matching value and pose transformation value ΔT between the current point cloud data and each keyframe point cloud data. Specifically, one point in the point cloud data is used directly. When the contour shape formed by the points of the two point cloud data has a high degree of overlap, it indicates a match, representing two point cloud data at the same location. This process is time-consuming, so it is preferred to obtain the most likely keyframes through frame signature matching, and then further process them to greatly reduce the amount of data processed and improve computational efficiency.

[0092] For step S104, the key frame with the largest matching pair is determined as the target key frame. Through the above steps, the ICP method is used to calculate the pose transformation value ΔT between the current point cloud data and the target key frame. From the above preset file, the pose T corresponding to the target key frame is determined. i , in position T i Based on this, the target pose T is obtained by superimposing the transformation ΔT. final =T i *ΔT.

[0093] The pose refers to the position and direction of the LiDAR on the map at the moment when the LiDAR collects point cloud data. The LiDAR may be in a different position each time it is turned on. For example, the position of the point cloud data this time differs from the position of a key frame in the point cloud map by 2 meters. The frame signature difference processing method determines that the current power-on position is close to the key frame position, but it may not completely overlap. Therefore, the 2m difference needs to be calculated through the ICP point cloud registration method. After that, the 2m difference needs to be supplemented to determine the current accurate position of the LiDAR on the map, so as to complete the repositioning, and then the autonomous driving can continue based on the current position.

[0094] See the overall implementation diagram for Figure 2 As shown, a point cloud map is generated through multi-frame point cloud data collected by the laser radar of one or more terminals, candidate key frames are screened, and the frame signature and pose of each key frame are calculated. When the terminal currently needs to be positioned, the laser radar is also used to collect the current frame of point cloud data, determine its frame signature, and calculate the difference between the frame signature and the frame signature of each key frame in the point cloud map to perform key frame screening. The matching degree and pose difference value of the current frame point cloud data and each screened key frame are calculated, and the pose of the key frame with the largest matching degree is adjusted to obtain the pose of the current terminal.

[0095] The method provided in the above embodiment can simply and effectively determine the candidate key frames for terminal relocation through a pre-constructed point cloud map, so as to screen the target key frames with the highest matching degree, realize the effective selection of relocation key frames, ensure the accuracy of the relocation results, and reduce the excessive consumption of computing resources and time, so that it can be applied to more complex environments.

[0096] See also Figure 3 , shows a schematic flow chart of an optional terminal relocation method according to an embodiment of the present invention, comprising the following steps:

[0097] S301: Convert the coordinate value of each point in the current frame point cloud data in the Cartesian coordinate system to the row index, column index, and distance between the projection point on the horizontal plane and the origin in the cylindrical coordinate system;

[0098] S302: Arrange all points according to row index and column index to generate two-dimensional data, calculate the inclination of two adjacent points in each column in turn, and mark the two adjacent points whose absolute value of the inclination is greater than or equal to a preset inclination threshold as vertical direction points;

[0099] S303: Counting the number of vertical points in each column to screen out one or more columns whose number is greater than or equal to a second preset number, and for each of the one or more columns, calculating the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of the vertical points;

[0100] S304: Determine the relative distance distribution of the vertical points between any two columns and the contribution value of each relative distance distribution, arrange the contribution values ​​in ascending order of the relative distance distribution to generate an array, and determine the generated array as a frame signature.

[0101] In the above implementation, for step S301, this implementation mainly describes how to use the frame signature processing program to process the point cloud data to obtain the frame signature. Take the 16-line laser radar as an example. The 16-line laser radar is composed of 16 laser emitters arranged in a row. These 16 laser emitters collect data once every 0.5 degrees of rotation, and a total of 16 rows and 1800 columns of data are obtained. In terms of effect, the more dimensions (beams) of the laser radar, the higher the measurement accuracy and the higher the safety.

[0102] LiDAR uses time of flight to calculate the distance of an object: when a laser pulse is emitted, its emission time and direction are recorded. The laser pulse propagates through the air until it hits an obstacle that reflects some of the energy. The sensor records the time of acquisition and reception. The spherical coordinates of the obstacle are calculated using the sensor's return time and the power received after each scan (as reflectivity).

[0103] The position of any target P in space can be determined by the following three coordinate values, as shown in Figure 4(a):

[0104] (1) Slant range of target R: the straight-line distance from the laser radar to the target.

[0105] (2) Azimuth (α): The angle between the projection of the target slant range R on the horizontal plane and a certain starting direction in the horizontal direction.

[0106] (3) Elevation angle (β): The angle between the target slant range R and its projection on the horizontal plane on the plumb plane.

[0107] The coordinate value of each point in the radar data in the Cartesian coordinate system is (x, y, z). For a multi-line laser radar, for example, 16-line radar data can be regarded as 16 rows and 1800 columns of data. Therefore, the coordinate value of each point needs to be converted to the cylindrical coordinate system to obtain the row and column index of each point and the distance from the projection point on the xy plane to the coordinate origin, see formula (1):

[0108]

[0109] The coordinates of point i are (x i ,y i ,z i ), ρ i is the distance from the projection point of point i on the xy plane to the origin of the coordinate system, row i is the row index of point i, columni It is the column index of point i, for example, the third row and fourth column. The vertical resolution of the 16-line laser radar is 2 degrees, which is used when calculating the row index. 8 is an empirical value representing the offset to ensure that the row index is a positive number. The horizontal resolution is 0.2 degrees, which is used when calculating column. If other multi-line laser radars are used, the values ​​here need to be adjusted adaptively.

[0110] For step S302, the point cloud data is arranged into two-dimensional data according to the row and column indexes, as shown in FIG4(b), which is a schematic diagram of two-dimensional data sorting. Take out each column of data, and for the inclination angle α of two adjacent points, the absolute value of the inclination angle α exceeds the preset inclination threshold α. max Two adjacent points (such as π / 3) are marked as vertical points.

[0111] Refer to formula (2), which is the formula for calculating the inclination angle α between two adjacent points (i-1) and i, where the coordinates of point (i-1) are (x i-1 ,y i-1 ,z i-1 ), the coordinates of point i are (x i ,y i ,z i ):

[0112]

[0113] For step S303, the number of vertical points in each column is counted M. For a column of data, if the number of vertical points M is greater than or equal to the preset number M σ (i.e., the second preset number, such as 5), then the following formula (3) is used to first calculate the average distance from the projection point of each column of vertical points on the xy plane to the coordinate origin: Afterwards based on Calculate the distance variance σ of each column of vertical points, but if M is less than M σ , then skip the column.

[0114]

[0115] In this way, we can find a column of points on an obstacle (such as a wall or a pole), and form a vertical line from this column of points. At the same time, we can use formula (4) to calculate the mean of the coordinates of the vertical points in each column in the X-axis and Y-axis directions, that is, the x and y mean:

[0116]

[0117] For step S304, the number M of points in the vertical direction is greater than or equal to the preset number M σAll columns are sorted by variance to select N (i.e., the third preset number, such as 20, and must be less than the number of these columns) target columns of data with the smallest variance. To ensure that the selected columns are not concentratedly distributed and to ensure the validity of the point cloud data, assuming that a column of data is selected, the n columns of data adjacent to the left and right of the column will no longer be selected. For example, the 50th column of data is hit on the wall, and the calculated variance is relatively small, so it is selected. Correspondingly, the variances obtained in the 40th to 49th and 50th to 59th columns are likely to be small, but they cannot be selected, and features need to be found in other positions, such as trees and poles. For the 40th to 59th column data mentioned above, they are adjacent data, and you can only select the column with the smallest variance, or you can select the middle column data, such as the 50th column data.

[0118] After selecting the N columns of data with the smallest variance, the relative distance distribution dist and the square of the variance σ between any two columns of data are calculated according to the following formula (5): 2 Any two columns here refer to kinds of permutations and combinations.

[0119]

[0120] According to the normal distribution standard formula (6), the contribution value Y of each relative distance distribution dist to the histogram is calculated, and the histogram of the distance distribution shown in Figure 4(c) is accumulated, and each relative distance distribution only takes the value of X in the range of (σ-dist) to (σ+dist) for calculation; where the histogram is a two-dimensional statistical chart, and the two coordinates are the statistical sample and the measurement of a certain attribute corresponding to the sample.

[0121]

[0122] According to the contribution value of each relative distance distribution in the histogram of the distance distribution shown in Figure 4(c), it is assumed that the cumulative contribution value when the relative distance distribution is 0-1 is 5, the cumulative contribution value when the relative distance distribution is 1-2 is 4, the cumulative contribution value when the relative distance distribution is 2-3 is 3, the cumulative contribution value when the relative distance distribution is 6-7 is 3, and the cumulative contribution value when the relative distance distribution is 7-8 is 4. The contribution values ​​are arranged in order from small to large relative distance distributions to obtain an array [5,4,3,0,0,0,3,4], which is the frame signature. Comparing the difference between two frame signatures is to compare whether the values ​​of each position in the two arrays are the same. Assuming that the frame signature of a key frame point cloud data in the point cloud map is [4,4,3,2,0,2,0,1], compared with the aforementioned [5,4,3,0,0,0,3,4], it can be seen that the values ​​of the second position, the third position and the fifth position are the same, and the values ​​of the remaining five positions are different.

[0123] It should be noted that the value 0 in the array cannot be removed. For example, after removing the value 0 in the array, the two frame signatures mentioned above become [5,4,3,3,4] and [4,4,3,2,2,1]. Although they can be compared based on the actual positions later, the calculation difficulty is undoubtedly increased compared to the previous comparison process. Therefore, in this solution, even if the contribution value of a relative distance distribution is 0, its position in the array must still be retained and the value must be set to 0.

[0124] In addition, the measurement units of the frame signatures are all set uniformly. For example, if the measurement unit is set to 1, the relative distance distribution is all based on 1, and the relative distance distribution is 0-1, 1-2, and 2-3. However, if the measurement unit is set to 2, the relative distance distribution is based on 2, and the relative distance distribution is 0-2, 2-4, and 4-6. The specific setting is unified according to the actual situation.

[0125] The overall implementation process can be found in Figure 5 As shown, the point cloud data is converted from the Cartesian coordinate system to the cylindrical coordinate system to obtain the row and column indexes and distances. The point cloud two-dimensional data is generated based on the row and column indexes. For each column of data, the inclination of two adjacent points is calculated to screen the vertical points, and then the distance variance, x average value and y average value of the vertical points are calculated. One or more columns with smaller variance are screened, and the relative distance distribution and contribution value of any two columns are calculated to generate a frame signature.

[0126] The method provided in the above embodiment first generates point cloud two-dimensional data according to row and column indices, counts vertical points and variances by column, selects columns with smaller variances to calculate relative distance distribution to reduce the amount of calculation and ensure that the obtained relative position is rotationally invariant. The contribution value of each relative distance distribution is accumulated to generate a histogram, and the point cloud data frame signature is obtained to enhance robustness and anti-interference.

[0127] See also Figure 6 , shows a schematic diagram of main modules of a terminal relocation device 600 provided in an embodiment of the present invention, including:

[0128] The frame signature module 601 is used to obtain the current frame point cloud data collected by the terminal in response to the relocation operation of the terminal, and perform signature processing on the current frame point cloud data to obtain the current frame signature;

[0129] The first calculation module 602 is used to obtain the frame signature of each key frame point cloud data in the point cloud map, calculate the difference value between the current frame signature and each frame signature, and determine a first preset number of key frame point cloud data in the order of the difference value from small to large;

[0130] The second calculation module 603 is used to determine the posture of each key frame point cloud data in the first preset number of key frame point cloud data, and calculate the matching value and posture transformation value of each key frame point cloud data and the current frame point cloud data;

[0131] The posture module 604 is used to superimpose the posture of the key frame point cloud data with the greatest matching degree on the corresponding posture transformation value to obtain the current target posture of the terminal in the point cloud map.

[0132] The implementation device of the present invention also includes a key frame module, which is used to:

[0133] During the operation of one or more terminals, multiple frames of point cloud data collected by the laser radar are obtained to construct a point cloud map based on the multiple frames of point cloud data;

[0134] According to the distances between adjacent frames of point cloud data in the point cloud map, key frame point cloud data whose distances meet a preset distance range are selected;

[0135] Perform signature processing on each key frame point cloud data to obtain a frame signature of each key frame point cloud data; and use a preset algorithm to calculate the pose of each key frame point cloud data.

[0136] In the implementation device of the present invention, the frame signature module 601 is used to:

[0137] Convert the coordinate value of each point in the Cartesian coordinate system of the current frame point cloud data into a row index, a column index, and a distance between the projection point on the horizontal plane and the origin in a cylindrical coordinate system;

[0138] Arrange all points according to row index and column index to generate two-dimensional data, calculate the inclination of two adjacent points in each column in turn, and mark the two adjacent points whose absolute value of the inclination is greater than or equal to the preset inclination threshold as vertical direction points;

[0139] Calculate the distance variance, horizontal axis coordinate mean and vertical axis coordinate mean of each column of vertical points to determine the relative distance distribution of vertical points between any two columns and the contribution value of each relative distance distribution;

[0140] The contribution values ​​are arranged in ascending order of relative distance distribution to generate an array, and the generated array is determined as a frame signature.

[0141] In the implementation device of the present invention, the frame signature module 601 is used to:

[0142] Counting the number of vertical points in each column to filter out one or more columns whose number is greater than or equal to a second preset number;

[0143] For each of the one or more columns, the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of the points in the vertical direction are calculated.

[0144] In the implementation device of the present invention, the frame signature module 601 is used to:

[0145] Determine a third preset number of columns in the order of distance variance from small to large;

[0146] The relative distance distribution between any two columns in the second preset number of columns is calculated according to the distance variance, the horizontal axis coordinate mean and the vertical axis coordinate mean of each column of vertical direction points.

[0147] In the implementation device of the present invention, the frame signature module 601 is used to:

[0148] For a plurality of adjacent columns of data in the third preset number of columns, a column of data having the smallest variance, or a column of data located in the middle of the plurality of columns of data is selected.

[0149] In the implementation device of the present invention, the frame signature module 601 is used to:

[0150] The contribution values ​​of the same relative distance distribution are accumulated, and a statistical chart is constructed based on the relative distance distribution and the accumulated contribution values. The contribution values ​​are arranged in ascending order according to the relative distance distribution in the statistical chart.

[0151] In addition, the specific implementation content of the device described in the embodiment of the present invention has been described in detail in the method described above, so the repeated content will not be described again here.

[0152] Figure 7 An exemplary system architecture 700 to which embodiments of the present invention may be applied is shown, including terminal devices 701 , 702 , 703 , a network 704 and a server 705 (only an example).

[0153] Terminal devices 701, 702, 703 can be various electronic devices with display screens and supporting web browsing, and various communication client applications are installed. Users can use terminal devices 701, 702, 703 to interact with server 705 through network 704 to receive or send messages, etc.

[0154] The network 704 is used to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0155] The server 705 may be a server that provides various services. It should be noted that the method provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the device is generally set in the server 705 .

[0156] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0157] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0159] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0160] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0161] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0162] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be arranged in a processor, for example, they may be described as: a processor including a frame signature module, a first calculation module, a second calculation module, and a posture module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the posture module may also be described as a "posture adjustment module".

[0164] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes any of the above terminal relocation methods.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A terminal relocation method, characterized in that: include: In response to the repositioning operation of the terminal, the current frame point cloud data collected by the terminal is obtained, and the current frame point cloud data is signed to obtain the current frame signature, including: converting the coordinate value of each point in the current frame point cloud data in the Cartesian coordinate system into a row index, a column index, and a distance between the projection point on the horizontal plane and the origin in the cylindrical coordinate system; arranging all points according to the row index and the column index to generate two-dimensional data, calculating the inclination of two adjacent points in each column in turn, and marking the two adjacent points whose absolute value of the inclination is greater than or equal to a preset inclination threshold as vertical points; calculating the distance variance, the mean of the horizontal axis coordinates, and the mean of the vertical axis coordinates of each column of vertical points to determine the relative distance distribution of the vertical points of each arbitrary two columns and the contribution value of each relative distance distribution; arranging the contribution values ​​in order from small to large according to the relative distance distribution to generate an array, and determining the generated array as the frame signature; Obtaining a frame signature of each key frame point cloud data in the point cloud map, calculating a difference value between the current frame signature and each frame signature, and determining a first preset number of key frame point cloud data in ascending order of the difference value; Determine the pose of each key frame point cloud data in the first preset number of key frame point cloud data, and calculate the matching value and pose transformation value of each key frame point cloud data and the current frame point cloud data; The pose of the key frame point cloud data with the greatest matching degree is superimposed with the corresponding pose transformation value to obtain the current target pose of the terminal in the point cloud map.

2. The method according to claim 1, characterized in that: Before the operation of relocating the terminal in response to the above, the method further comprises: During the operation of one or more terminals, multiple frames of point cloud data collected by the laser radar are obtained to construct a point cloud map based on the multiple frames of point cloud data; According to the distances between adjacent frames of point cloud data in the point cloud map, key frame point cloud data whose distances meet a preset distance range are selected; Perform signature processing on each key frame point cloud data to obtain a frame signature of each key frame point cloud data; and use a preset algorithm to calculate the pose of each key frame point cloud data.

3. The method according to claim 1, characterized in that The calculation of the distance variance, the horizontal axis coordinate mean and the vertical axis coordinate mean of each column of vertical direction points includes: Counting the number of vertical points in each column to filter out one or more columns whose number is greater than or equal to a second preset number; For each of the one or more columns, the distance variance, the horizontal axis coordinate mean, and the vertical axis coordinate mean of the points in the vertical direction are calculated.

4. The method according to claim 3, characterized in that The determining the relative distance distribution between any two columns comprises: Determine a third preset number of columns in the order of distance variance from small to large; The relative distance distribution between any two columns in the second preset number of columns is calculated according to the distance variance, the horizontal axis coordinate mean and the vertical axis coordinate mean of each column of vertical direction points.

5. The method according to claim 4, characterized in that The determining of the third preset number of columns further includes: For a plurality of adjacent columns of data in the third preset number of columns, a column of data having the smallest variance, or a column of data located in the middle of the plurality of columns of data is selected.

6. The method according to claim 1, characterized in that The contribution values ​​are arranged in ascending order according to the relative distance distribution, including: The contribution values ​​of the same relative distance distribution are accumulated, and a statistical chart is constructed based on the relative distance distribution and the accumulated contribution values. The contribution values ​​are arranged in order from small to large according to the relative distance distribution in the statistical chart.

7. A terminal relocation device, characterized in that: include: A frame signature module is used to respond to the repositioning operation of the terminal, obtain the current frame point cloud data collected by the terminal, perform signature processing on the current frame point cloud data, and obtain the current frame signature, including: converting the coordinate value of each point in the current frame point cloud data in the Cartesian coordinate system into a row index, a column index, and a distance between the projection point on the horizontal plane and the origin in the cylindrical coordinate system; arranging all points according to the row index and the column index to generate two-dimensional data, calculating the inclination of two adjacent points in each column in turn, and marking two adjacent points whose absolute value of the inclination is greater than or equal to a preset inclination threshold as vertical points; calculating the distance variance, the mean of the horizontal axis coordinates, and the mean of the vertical axis coordinates of each column of vertical points to determine the relative distance distribution of the vertical points between any two columns and the contribution value of each relative distance distribution; arranging the contribution values ​​in order from small to large according to the relative distance distribution to generate an array, and determining the generated array as the frame signature; A first calculation module is used to obtain a frame signature of each key frame point cloud data in the point cloud map, and calculate a difference value between the current frame signature and each frame signature, so as to determine a first preset number of key frame point cloud data in ascending order of the difference value; A second calculation module is used to determine the posture of each key frame point cloud data in the first preset number of key frame point cloud data, and calculate the matching value and posture transformation value of each key frame point cloud data and the current frame point cloud data; The posture module is used to superimpose the posture of the key frame point cloud data with the greatest matching degree on the corresponding posture transformation value to obtain the current target posture of the terminal in the point cloud map.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images