Global map-based relocation method, mobile device and storage medium
By selecting candidate solution locations in the global map and generating simulated keyframes for similarity matching, the existing relocation methods have solved the problems of low accuracy and high maintenance workload in indoor environments, and efficient robot relocation is achieved.
Patent Information
- Application Number
- CN202410965482.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-26
AI Technical Summary
The existing descriptor-based relocation method has low relocation accuracy and high maintenance work in indoor environments. The brute-force search method takes too long, affecting the efficiency of robot tasks.
Using a relocation method based on the global map, the positioning sensor performs coarse position in the global map, selects candidate solution locations, simulates perception data, generates simulated keyframes, and performs similarity matching to achieve relocation.
Improve the success rate and accuracy of relocation, reduce map maintenance workload and memory usage, and improve task efficiency.
Smart Images

Figure CN119025929A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of robotics technology, and in particular to a global map-based repositioning method, a removable device, and a storage medium. Background Art
[0002] Obtaining high-precision positioning results is one of the prerequisites for the robot to successfully perform tasks. Since robots are usually faced with performing tasks in a known environment, they usually pre-construct a global map of the task scene, and then perform matching and positioning based on the global map. For matching and positioning methods based on prior maps, how to determine the initial position of the robot is a crucial link. This step is usually called the robot's relocalization process. In open scenes such as outdoors, the Global Navigation Satellite System (GNSS) can be relied on to assist in relocalization. However, in scenes such as indoors, relocalization can only rely on sensors such as lidar and visual sensors carried by the robot. For example, for lidar, relocalization methods can generally be divided into two types: relocalization methods based on brute force search and relocalization methods based on descriptors.
[0003] Although the relocalization method based on brute force search has higher relocalization accuracy, it takes too long to locate the object, which will greatly reduce the efficiency of the robot in performing tasks. Although the traditional relocalization method based on descriptors can reduce the time consumption for positioning, it relies on pre-collected key frame data for descriptor generation. If the pre-collected key frame data is incomplete, the relocalization accuracy will be reduced, and the pre-collected key frame data needs to be maintained, which increases the maintenance workload. Summary of the invention
[0004] In order to solve the above technical problems, the present application at least provides a relocation method based on a global map, a removable device and a storage medium.
[0005] In a first aspect, the present application provides a global map-based repositioning method, which is applied to a movable device, wherein the movable device is deployed with a positioning sensor, and the positioning sensor is used to collect scene perception data of a scene to be positioned. The method comprises: obtaining a global map corresponding to the scene to be positioned and device parameters for the positioning sensor to collect perception data, and roughly positioning the movable device to obtain an initial position of the positioning sensor in the global map; selecting a candidate solution position in the global map based on the initial position to obtain a set of candidate solution positions; using the device parameters, simulating the perception data that can be collected by the positioning sensor at each candidate solution position in the global map to obtain a simulated key frame corresponding to each candidate solution position; performing similarity matching between a current key frame currently collected by the positioning sensor and a simulated key frame, and selecting a successfully matched simulated key frame from multiple simulated key frames; repositioning the movable device based on the successfully matched simulated key frame to obtain a repositioning result.
[0006] In one embodiment, candidate solution positions are selected in a global map based on an initial position to obtain a set of candidate solution positions, including: obtaining a search range by dividing the global map based on the initial position; and selecting candidate solution positions from the search range according to a preset search interval to obtain a set of candidate solution positions.
[0007] In one embodiment, the positioning sensor includes a laser radar; using device parameters, the perception data that can be collected by the positioning sensor at each candidate solution position in the global map is simulated to obtain a simulated key frame corresponding to each candidate solution position, including: using the device parameters to project the global map into a distance image centered on the candidate solution position to obtain the perception data that can be collected at the candidate solution position; based on the perception data that can be collected at the candidate solution position, a simulated key frame corresponding to each candidate solution position is obtained.
[0008] In one embodiment, the device parameters include a ranging range of a positioning sensor; using the device parameters, a global map is projected into a distance image centered on a candidate solution position to obtain perception data that can be collected by the candidate solution position, including: extracting point cloud data in the global map centered on the candidate solution position and within the ranging range to obtain a local submap; based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, each point in the local submap is projected into a distance image centered on the candidate solution position; and the points stored in the projected distance image are used as perception data that can be collected by the candidate solution position.
[0009] In one embodiment, based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, each point in the local submap is projected into a distance image centered on the candidate solution position, including: calculating the scanning angle and scanning distance of the point in the local submap relative to the candidate solution position based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap; using the scanning angle, calculating the index of the point in the local submap in the distance image to be projected; combining the index and the scanning distance to obtain the projection value of the point in the local submap in the distance image.
[0010] In one embodiment, the method further includes: if there are points with the same index in the local sub-map, retaining the point with the smallest scanning distance in the distance image and deleting other points.
[0011] In one embodiment, the current key frame currently acquired by the positioning sensor is matched with the simulated key frame, and a successfully matched simulated key frame is selected from multiple simulated key frames, including: performing feature encoding processing on the simulated key frame and the current key frame respectively to obtain descriptors corresponding to the simulated key frame and the current key frame; calculating the distance between the descriptors of the simulated key frame and the current key frame; and taking the simulated key frame whose distance meets the preset conditions as the successfully matched simulated key frame.
[0012] In one embodiment, the simulated key frame whose distance meets the preset condition is used as the successfully matched simulated key frame, including: selecting the simulated key frame with the smallest distance between the descriptor of the current key frame and multiple simulated key frames as the successfully matched simulated key frame.
[0013] The second aspect of the present application provides a repositioning device based on a global map, which is deployed on a movable device. The movable device is equipped with a positioning sensor, which is used to collect scene perception data of the scene to be positioned. The device includes: a data acquisition module, which is used to obtain the global map corresponding to the scene to be positioned and the device parameters of the positioning sensor for collecting perception data, and roughly locate the movable device to obtain the initial position of the positioning sensor in the global map; a candidate solution selection module, which is used to select candidate solution positions in the global map based on the initial position to obtain a set of candidate solution positions; a key frame simulation module, which is used to use the device parameters to simulate the perception data that can be collected by the positioning sensor at each candidate solution position in the global map, and obtain the simulated key frames corresponding to each candidate solution position; a key frame matching module, which is used to perform similarity matching between the current key frame currently collected by the positioning sensor and the simulated key frame, and select the successfully matched simulated key frame from multiple simulated key frames; a repositioning module, which is used to reposition the movable device based on the successfully matched simulated key frame to obtain a repositioning result.
[0014] A third aspect of the present application provides a removable device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned global map-based repositioning method.
[0015] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned global map-based relocation method when executed by a processor.
[0016] The above scheme obtains the global map corresponding to the scene to be positioned and the device parameters of the positioning sensor for perceptual data collection, and roughly locates the movable device to obtain the initial position of the positioning sensor in the global map; based on the initial position, the candidate solution position is selected in the global map to obtain a set of candidate solution positions; using the device parameters, the perceptual data that can be collected by the positioning sensor at each candidate solution position in the global map is simulated to obtain the simulated key frames corresponding to each candidate solution position, thereby expanding the range of candidate solutions and improving the success rate of relocation. In addition, the simulated key frames are generated in real time based on the matching results, which reduces memory usage and further reduces the workload of map maintenance. Then, the current key frame currently collected by the positioning sensor and the simulated key frame are matched for similarity, and the successfully matched simulated key frame is selected from multiple simulated key frames. The movable device is relocated based on the successfully matched simulated key frame to obtain an accurate relocation result.
[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0019] Figure 1 is a schematic diagram of rematching in the related art;
[0020] Figure 2 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;
[0021] Figure 3 is a flow chart of a global map-based relocation method shown in an exemplary embodiment of the present application;
[0022] Figure 4 is a block diagram of a global map-based repositioning device shown in an exemplary embodiment of the present application;
[0023] Figure 5 is a schematic structural diagram of a movable device shown in an exemplary embodiment of the present application;
[0024] Figure 6 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0025] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0026] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0027] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.
[0028] The traditional descriptor-based relocalization method first needs to collect the key frame data generated during the scanning process, generate the corresponding descriptor database based on the key frame data, and search the descriptor database based on the descriptor generated by the current frame point cloud in the relocalization stage, and select the most similar descriptor in the descriptor database as the relocalization result.
[0029] However, the traditional descriptor-based relocalization method has at least two defects: 1. Since the descriptors generated based on the keyframes are usually sensitive to the position, the descriptors generated at different positions are completely different, and the robot cannot be relocalized at a distance from the keyframe position, such as Figure 1 As shown in the figure, the robot collects key frames along the scanning route. If the robot is far away from the scanning route during the relocalization stage, it cannot match the pre-collected key frames. Therefore, in order to ensure that the robot can be relocated successfully when performing tasks, it is necessary to ensure that the pre-collected key frames can cover all task routes of the robot. This will result in the need to walk through too many repeated routes during the scanning process, increasing the scanning workload; 2. Since the scenarios in which the robot performs tasks are usually faced with environmental changes, it is necessary to regularly maintain the global map. The method of using key frames to generate descriptors will further increase the workload of map maintenance and occupy a large amount of memory resources.
[0030] In order to solve the above technical problems, the present application provides a global map-based relocation method, a removable device and a storage medium.
[0031] The following is a description of the global map-based relocation method provided in an embodiment of the present application.
[0032] Please refer to Figure 2 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a mobile device 210 and a server 220, and the mobile device 210 and the server 220 are connected to each other for communication.
[0033] Among them, the movable device 210 can be a sweeping robot, a freight robot, etc., and this application does not limit the type of the movable device.
[0034] In some embodiments, the movable device 210 includes a mobile chassis, a photoelectric ranging sensor, an image sensor, a lidar sensor, etc., wherein the mobile chassis includes a motion controller, a motor, a battery, an embedded computer, an odometer, etc.
[0035] Server 220 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.
[0036] Exemplarily, the mobile device 210 pre-stores a global map of the scene to be positioned, and the global map stores the locations of obstacles, scene building structures of the scene to be positioned, such as the locations of shelves, workbenches, house support frames, and goods to be transported. The mobile device 210 performs a point cloud scan of the current environment through a positioning sensor, and repositions according to the point cloud scan results to obtain the current position of the mobile device 210 in the global map, and performs subsequent tasks according to the current position obtained by the repositioning, such as map navigation, cargo handling, and other tasks.
[0037] In the multi-sensor based positioning and navigation method provided in the embodiment of the present application, the execution subject of each step can be the mobile device 210, such as the client of the target application installed and running in the mobile device 210, or the server 220, or the mobile device 210 and the server 220 interact and cooperate to execute, that is, part of the steps of the method are executed by the mobile device 210 and the other part of the steps are executed by the server 220.
[0038] See also Figure 3 , Figure 3is a flowchart of a global map-based relocation method shown in an exemplary embodiment of the present application. The global map-based relocation method can be applied to Figure 2 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.
[0039] like Figure 3 As shown, the global map-based relocation method includes at least steps S310 to S350, which are described in detail as follows:
[0040] Step S310: obtaining a global map corresponding to the scene to be positioned and device parameters of the positioning sensor for sensing data collection, and performing rough positioning on the movable device to obtain an initial position of the positioning sensor in the global map.
[0041] Obtaining a global map corresponding to the scene to be located. The mobile device may perform point cloud scanning on the scene to be located in advance, and generate a global map corresponding to the scene to be located based on the point cloud data obtained by the scan; or other devices may perform point cloud scanning on the scene to be located and construct a generated global map, and then send it to the mobile device, which is not limited in this application.
[0042] Obtain the device parameters of the positioning sensor for collecting perception data. The positioning sensor is used to collect scene perception data of the scene to be positioned. The positioning sensor includes but is not limited to laser radar, camera, etc. If the positioning sensor is a laser radar, the global map is a point cloud map, and the perception data is point cloud data. If the positioning sensor is a camera, the global map is a visual map, and the perception data is image data. The device parameters include but are not limited to the resolution, ranging range, wavelength and other parameters of the positioning sensor.
[0043] In addition, the movable device is roughly positioned to obtain the initial position of the positioning sensor in the global map.
[0044] Exemplarily, it is possible to obtain pre-input positioning parameters and use the positioning parameters as the initial position of the positioning sensor in the global map; it is also possible to obtain the initial position of the positioning sensor in the global map based on technologies such as Bluetooth, mobile hotspot (Wi-Fi), and near field communication (NFC). This application does not limit the specific implementation method of coarse positioning of movable devices.
[0045] Since the location information obtained by coarse positioning is inaccurate, further precise positioning is required.
[0046] Step S320: Select candidate solution positions in the global map based on the initial position to obtain a set of candidate solution positions.
[0047] According to the initial position, candidate solution positions are selected in the global map to obtain one or more candidate solution positions, and the selected candidate solution positions are added to the candidate solution position set.
[0048] Exemplarily, a search range is divided in the global map according to the initial position, and a candidate solution position is selected within the search range.
[0049] For example, a search interval is obtained, and candidate solution positions are selected according to the search interval. At this time, the number of candidate solution positions selected is proportional to the size of the search range. Among them, a fixed search interval can be used within a search range; or multiple different search intervals can be used within a search range. For example, the closer the distance to the initial position, the smaller the search interval, and the farther the distance to the initial position, the larger the search interval. Since the probability of successful key frame matching corresponding to the candidate solution position closer to the initial position is higher, the accuracy of positioning can be guaranteed while reducing the number of simulated key frames generated subsequently.
[0050] The search interval may be a preset value or a flexibly calculated value, such as calculating the search interval based on the positioning accuracy level corresponding to the task to be performed, the obstacle distribution in the area where the initial position is located, the rate of dynamic change of the environment in the area where the initial position is located, etc., which is not limited in this application.
[0051] For another example, the number of candidate solution positions to be selected is obtained, and a corresponding number of candidate solution positions are selected within the search range according to the number of candidate solution positions to be selected. For example, a corresponding number of candidate solution positions are randomly selected within the search range according to the number of candidate solution positions to be selected; or, the search interval of the search range is calculated according to the number of candidate solution positions to be selected, and a corresponding number of candidate solution positions are selected according to the search interval. Among them, the number of candidate solution positions to be selected can be pre-set, or can be flexibly calculated according to actual conditions. For example, the number of candidate solution positions is calculated according to the positioning accuracy level corresponding to the task to be performed, the obstacle distribution in the area where the initial position is located, the dynamic change rate of the environment in the area where the initial position is located, etc. For example, the higher the positioning accuracy level corresponding to the task to be performed, the more obstacles there are in the area where the initial position is located, and the greater the dynamic change rate of the environment in the area where the initial position is located, the larger the number of candidate solution positions, otherwise the smaller the number of candidate solution positions.
[0052] Exemplarily, the search range may not be divided, and the number of candidate solution positions to be selected and the search interval may be directly obtained. The initial position may be used as the selection starting point, and a corresponding number of candidate solution positions may be selected according to the number of candidate solution positions to be selected and the search interval.
[0053] It can be understood that in addition to the above-mentioned methods for selecting candidate solution positions, candidate solution positions can also be selected according to other methods. The specific selection method can be flexibly adjusted according to the actual application situation, such as analyzing the scene complexity within the preset range of the initial position (such as determining the scene complexity based on the number and distribution of obstacles, ground flatness, etc.), and determining the method for selecting the candidate solution position based on the scene complexity. This application does not limit this.
[0054] Step S330: using the device parameters, simulating the perception data that can be collected by the positioning sensor at each candidate solution position in the global map, and obtaining a simulated key frame corresponding to each candidate solution position.
[0055] After obtaining the candidate solution positions, the corresponding device parameters of the positioning sensor for perception data collection can be used to simulate the perception data (such as point cloud data, image data) that can be collected by the positioning sensor at the candidate solution positions of the global map, thereby generating simulated key frames corresponding to each candidate solution position.
[0056] Exemplarily, based on the device parameters corresponding to the perception data collection of the positioning sensor, the mapping function used to map the midpoint of the global map to the simulation key frame corresponding to the candidate solution position can be known. Thus, based on the pose relationship and mapping function between the midpoint of the global map and the candidate solution position, the midpoint of the global map is converted into point cloud data that can be collected by the positioning sensor at the candidate solution position, and the simulation key frame corresponding to the candidate solution position is obtained.
[0057] Exemplarily, a keyframe simulation model may be pre-built, and by inputting the global map, the candidate solution position and the device parameters into the keyframe simulation model, a simulated keyframe output by the keyframe simulation model may be obtained. The keyframe simulation model may be built based on a neural network, and may be obtained by training the keyframe simulation model based on sample data carrying sample labels, and the sample labels may be real keyframes collected by the movable device at the candidate solution position.
[0058] It is understandable that in addition to the above-mentioned examples of generating simulated key frames, other methods can also be used to generate simulated key frames. The specific selection method can be flexibly adjusted according to actual application conditions, and this application does not limit this.
[0059] Step S340: performing similarity matching between the current key frame currently acquired by the positioning sensor and the simulated key frame, and selecting a successfully matched simulated key frame from multiple simulated key frames.
[0060] The simulated key frame generated by the above steps is used as the key frame matching set corresponding to the current initial position, the current key frame currently collected by the positioning sensor is obtained, and the current key frame currently collected by the positioning sensor and the simulated key frame are matched for similarity to obtain a successfully matched simulated key frame.
[0061] For example, taking the positioning sensor as a laser radar, point cloud matching is performed on the current key frame and the simulated key frame. The point cloud matching algorithm used includes but is not limited to the iterative closest point (ICP) algorithm, the normal distribution transform (NDT) algorithm, the generalized ICP (GICP) algorithm, etc., which is not limited in this application.
[0062] It should be noted that the number of successfully matched simulation key frames can be one or more, and this application does not limit this.
[0063] Step S350: repositioning the movable device based on the successfully matched simulated key frames to obtain a repositioning result.
[0064] The movable device is relocated according to the successfully matched one or more frames of simulated key frames to estimate the position and posture of the movable device and obtain a relocation result.
[0065] Exemplarily, the pose transformation parameters between the successfully matched simulation key frame and the current key frame are calculated, and the repositioning result is obtained based on the pose and pose transformation parameters corresponding to the successfully matched simulation key frame.
[0066] For example, the posture of the movable device at the current moment predicted by the successfully matched simulated key frame is calculated based on the posture transformation parameters as the initial matching value, and then the current key frame is matched with the global map using the normal distribution transformation algorithm, iterative closest point algorithm, etc. to optimize the initial matching value and obtain the final relocalization result.
[0067] Of course, other algorithms can also be used to use the successfully matched simulation key frames to reposition the movable device to obtain a repositioning result. For example, when there are multiple successfully matched simulation key frames, the posture corresponding to each successfully matched simulation key frame can be combined to solve the final repositioning result. This application does not limit this.
[0068] Next, some embodiments of the present application are described by way of example.
[0069] In some implementations, taking the positioning sensor including a laser radar as an example, in step S330, the device parameters are used to simulate the perception data that can be collected by the positioning sensor at each candidate solution position in the global map, and the simulated key frames corresponding to each candidate solution position are obtained, including:
[0070] Step S331: Using device parameters, project the global map into a range image centered on the candidate solution position to obtain the perception data that can be collected at the candidate solution position.
[0071] The positioning sensor is a laser radar, and the perception data corresponding to the positioning sensor is the point cloud data obtained by scanning the point cloud of the positioning environment. Among them, the range image refers to the two-dimensional range image (Range lmages) obtained by converting the three-dimensional laser point cloud through spherical projection (SphericalProjection).
[0072] Exemplarily, the device parameters include the ranging range of the positioning sensor; using the device parameters, the global map is projected into a distance image centered on the candidate solution position to obtain perception data that can be collected by the candidate solution position, including: extracting point cloud data in the global map centered on the candidate solution position and within the ranging range to obtain a local submap; based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, each point in the local submap is projected into the distance image centered on the candidate solution position; and the points stored in the projected distance image are used as perception data that can be collected by the candidate solution position.
[0073] For example, the lidar point cloud within a radius of r centered at the candidate solution position in the global map is extracted to obtain a local submap.
[0074] According to the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, each point in the local submap is projected into the range image centered on the candidate solution position.
[0075] Specifically, based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, the scanning angle and scanning distance of the points in the local submap relative to the candidate solution position are calculated; the index of the points in the local submap in the distance image to be projected is calculated using the scanning angle; and the projection value of the points in the local submap in the distance image is obtained by combining the index and the scanning distance.
[0076] For example, the formula for calculating the scanning angle and scanning distance of a point in a local submap relative to the candidate solution position can refer to the following formulas 1 to 3:
[0077] Formula 1:
[0078] Formula 2:
[0079] Formula 3:
[0080] Among them, r i Indicates the scanning distance; θ i Indicates the vertical angle in the scanning angle; Indicates the horizontal angle in the scanning angle; (p ix ,p iy ,p iz ) represents the point p in the local submap i Point cloud coordinates; (x i ,y i ,z i ) represents the candidate solution position t i The point cloud coordinates.
[0081] Then, the scanning angle is used to calculate the index of the point in the local sub-map in the range image to be projected. The relevant calculation formulas can be found in the following formulas 4 to 7:
[0082] Formula 4:
[0083] Formula 5: |θ b′ -θ i |=argmin(|θ k -θ i |)
[0084] Formula 6:
[0085] Formula 7:
[0086] Among them, floor means rounding down; represents the horizontal angle in the scanning angle; h is the horizontal resolution in the equipment parameters; θ i Indicates the vertical angle in the scanning angle; θ b′ b′ and θ in k The k in the figure represents the vertical angle subscript, which is obtained according to the vertical angle distribution set of the laser radar, such as the vertical angle distribution set of the laser radar {Q} = {θ0,θ1,...,θ m-1}, then the value range of b′ and k is {0,1,...,m-1}; argmin represents the parameter value of a function that takes the minimum value in its domain; represents the horizontal angle error; Δθ represents the vertical angle error; finally, the index (a pi , b pi ). Among them, a pi or b piWhen the value is -1, it means that the point is invalid.
[0087] Combining the index and the scan distance, we can get the projection value of the point in the local submap in the distance image. For example, the projection value is expressed as (a pi , b pi , r i ).
[0088] In some implementations, if there are points with the same index in the local sub-map, the point with the smallest scan distance is retained in the range image, and the other points are deleted.
[0089] Each point in the local submap is projected to obtain its projection value in the range image, and then the points stored in the range image are extracted to obtain the lidar point cloud simulated at the candidate solution position, that is, the perception data simulated at the candidate solution position is obtained.
[0090] Step S332: based on the perception data that can be collected at the candidate solution positions, obtain the simulated key frames corresponding to each candidate solution position.
[0091] The simulated key frame set is generated by simulation in the above embodiment, and the current key frame currently acquired by the positioning sensor is matched with each simulated key frame in the simulated key frame set to achieve repositioning of the movable device.
[0092] In some implementations, in step S340, matching the current key frame currently acquired by the positioning sensor with the simulated key frame, and selecting a successfully matched simulated key frame from multiple simulated key frames includes:
[0093] Step S341: performing feature encoding processing on the simulated key frame and the current key frame respectively to obtain descriptors corresponding to the simulated key frame and the current key frame.
[0094] The descriptor is used to describe the feature information of the key frame, for example, the SC (Scan Context) descriptor, the PFH (Point Feature Histogram) descriptor, etc. This application does not limit the type of descriptor used.
[0095] For example, taking the Scan Context descriptor as an example, the feature encoding process of the simulated key frame is illustrated as follows: the simulated key frame is divided into multiple point cloud units p(i,j), where i represents the i-th ring and j represents the j-th sector. The maximum height of the area is stored in each point cloud unit. Assume that N s and N r Represent the number of sectors and rings respectively, then we can get N s ×N r The descriptor matrix I i, thereby combining the descriptor matrices corresponding to all simulated key frames to obtain the descriptor set {I}.
[0096] It should be noted that the steps of feature encoding processing of the current key frame are the same as those of feature encoding processing of the simulation key frame, which will not be described in detail here.
[0097] Step S342: Calculate the distance between the descriptors of the simulated key frame and the current key frame, where the distance is used to reflect the similarity between the simulated key frame and the current key frame.
[0098] Step S343: The simulated key frame whose distance meets the preset condition is regarded as a successfully matched simulated key frame.
[0099] The simulated key frame whose distance meets the preset condition may be a simulated key frame whose distance is less than a preset distance threshold, or a simulated key frame with a minimum distance value, which is not limited in the present application.
[0100] An example is given in which a simulated key frame having the smallest descriptor distance with the current key frame among multiple simulated key frames is selected as a simulated key frame that is successfully matched.
[0101] For example, the current key frame is represented by S k , the descriptor corresponding to the current key frame is represented as I k , first evaluate the similarity between the descriptors of the simulated key frame and the current key frame, and the relevant calculation formula is shown in the following formula 8:
[0102] Formula 8:
[0103] in, I i and I k The corresponding column vectors at the same index, Respectively and The vector norm, d(I i ,I k ) represents the similarity between the descriptors of the simulated keyframe and the current keyframe.
[0104] According to the similarity evaluation result, the distance between the descriptor of the simulated key frame and the current key frame is calculated. The relevant calculation formulas are shown in the following formulas 9 and 10:
[0105] Formula 9:
[0106] Formula 10:
[0107] Among them, I i,n Denotes the descriptor matrix I iThe new descriptor matrix formed by shifting n columns to the right; [N s ] has a value of {1,2,...,N s-1 ,N s};n * Represents the translation matrix corresponding to the minimum distance.
[0108] By calculating the descriptor I in the descriptor set {I} that matches the current key frame k The descriptor with the smallest distance between i , according to I i The position and n * Get the matching initial value T init =(x init ,y init ,z init ,yaw init ) where (x init ,y init ,z init ) is the initial position, yaw angle yaw init =n * / N s .
[0109] Then, according to the matching initial value T init , the current key frame S k Match it with the global map to get the final relocalization result. Specifically, the current key frame S k Match it with the local submap centered on the matching candidate solution position in the global map to obtain the final relocalization result.
[0110] The global map-based repositioning method provided in the present application obtains the global map corresponding to the scene to be positioned and the device parameters of the positioning sensor for perceptual data collection, and roughly positions the movable device to obtain the initial position of the positioning sensor in the global map; based on the initial position, the candidate solution position is selected in the global map to obtain a set of candidate solution positions; using the device parameters, the perceptual data that can be collected by the positioning sensor at each candidate solution position in the global map is simulated to obtain the simulated key frames corresponding to each candidate solution position, thereby expanding the range of candidate solutions and improving the success rate of repositioning. In addition, the simulated key frames are generated in real time in the matching results, which reduces memory usage and further reduces the workload of map maintenance. Then, the current key frame currently collected by the positioning sensor and the simulated key frame are matched for similarity, and the successfully matched simulated key frame is selected from multiple simulated key frames. The movable device is repositioned based on the successfully matched simulated key frame to obtain an accurate repositioning result.
[0111] Figure 4: is a block diagram of a global map-based relocation device shown in an exemplary embodiment of the present application. The global map-based relocation device is deployed on a movable device. The movable device is equipped with a positioning sensor, which is used to collect scene perception data of the scene to be located. Figure 4 As shown, the exemplary global map-based relocation device 400 includes: a data acquisition module 410, a candidate solution selection module 420, a key frame simulation module 430, a key frame matching module 440 and a relocation module 450. Specifically:
[0112] The data acquisition module 410 is used to obtain the global map corresponding to the scene to be positioned and the device parameters of the positioning sensor for sensing data collection, and to roughly position the movable device to obtain the initial position of the positioning sensor in the global map;
[0113] A candidate solution selection module 420 is used to select candidate solution positions in the global map based on the initial position to obtain a set of candidate solution positions;
[0114] The key frame simulation module 430 is used to simulate the perception data that can be collected by the positioning sensor at each candidate solution position in the global map by using the device parameters, and obtain the simulated key frame corresponding to each candidate solution position;
[0115] The key frame matching module 440 is used to perform similarity matching between the current key frame currently acquired by the positioning sensor and the simulated key frame, and select a successfully matched simulated key frame from multiple simulated key frames;
[0116] The repositioning module 450 is used to reposition the movable device based on the successfully matched simulated key frames to obtain a repositioning result.
[0117] It should be noted that the global map-based relocation device provided in the above embodiment and the global map-based relocation method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the global map-based relocation device provided in the above embodiment can distribute the above functions to different functional modules as needed, i.e., divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.
[0118] See also Figure 5 , Figure 51 is a schematic diagram of the structure of an embodiment of a mobile device of the present application. The mobile device 500 includes a memory 501 and a processor 502, and the processor 502 is used to execute program instructions stored in the memory 501 to implement the steps in any of the above-mentioned global map-based relocation method embodiments. In a specific implementation scenario, the mobile device 500 may include, but is not limited to: a microcomputer, a server, and in addition, the mobile device 500 may also include a laptop computer, a tablet computer and other mobile devices, which are not limited here.
[0119] Specifically, the processor 502 is used to control itself and the memory 501 to implement the steps in any of the above-mentioned global map-based relocation method embodiments. The processor 502 can also be called a central processing unit (CPU). The processor 502 may be an integrated circuit chip with signal processing capabilities. The processor 502 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 502 can be implemented by an integrated circuit chip.
[0120] See also Figure 6 , Figure 6 The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor, and the program instructions 610 are used to implement the steps in any of the above-mentioned global map-based relocation method embodiments.
[0121] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0122] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0123] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0124] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.
Claims
1. A relocation method based on a global map, characterized in that: Applied to a mobile device, the mobile device is deployed with a positioning sensor, the positioning sensor is used to collect scene perception data of a scene to be positioned, including: Acquire a global map corresponding to the scene to be positioned and device parameters of the positioning sensor for sensing data collection, and roughly position the movable device to obtain an initial position of the positioning sensor in the global map; Based on the initial position, candidate solution positions are selected in the global map to obtain a set of candidate solution positions; Using the device parameters, simulating the perception data that can be collected by the positioning sensor at each candidate solution position in the global map, and obtaining a simulation key frame corresponding to each candidate solution position; Performing similarity matching between the current key frame currently acquired by the positioning sensor and the simulated key frame, and selecting a successfully matched simulated key frame from the multiple simulated key frames; The movable device is repositioned based on the successfully matched simulated key frame to obtain a repositioning result.
2. The method according to claim 1, characterized in that: The selecting of candidate solution positions in the global map based on the initial position to obtain a set of candidate solution positions includes: Dividing a search range from the global map based on the initial position; Candidate solution positions are selected from the search range according to a preset search interval to obtain a set of candidate solution positions.
3. The method according to claim 1, characterized in that The positioning sensor includes a laser radar; using the device parameters to simulate the perception data that can be collected by the positioning sensor at each candidate solution position in the global map to obtain a simulated key frame corresponding to each candidate solution position, including: Using the device parameters, projecting the global map into a range image centered on the candidate solution position to obtain perception data that can be collected by the candidate solution position; Based on the perception data that can be collected at the candidate solution positions, a simulation key frame corresponding to each candidate solution position is obtained.
4. The method according to claim 3, characterized in that The device parameters include the ranging range of the positioning sensor; and using the device parameters to project the global map into a range image centered on the candidate solution position to obtain the perception data that can be collected by the candidate solution position includes: Extracting point cloud data centered at the candidate solution position and within the ranging range from the global map to obtain a local submap; Based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap, projecting each point in the local submap into a range image centered on the candidate solution position; The points stored in the projected range image are used as the perception data that can be collected by the candidate solution position.
5. The method according to claim 4, characterized in that The projecting each point in the local submap into a range image centered at the candidate solution position based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap includes: Calculating a scanning angle and a scanning distance of a point in the local submap relative to the candidate solution position based on the point cloud coordinates of the candidate solution position and the point cloud coordinates of each point in the local submap; Using the scanning angle, calculating the index of the point in the local sub-map in the range image to be projected; The index and the scanning distance are combined to obtain projection values of points in the local sub-map in the range image.
6. The method according to claim 5, characterized in that The method further comprises: If there are points with the same index in the local sub-map, the point with the smallest scanning distance is retained in the range image, and other points are deleted.
7. The method according to claim 1, characterized in that The similarity matching between the current key frame currently acquired by the positioning sensor and the simulated key frame, and selecting a successfully matched simulated key frame from the multiple simulated key frames, includes: Performing feature encoding processing on the simulated key frame and the current key frame respectively to obtain descriptors corresponding to the simulated key frame and the current key frame; Calculating a distance between the descriptors of the simulated key frame and the current key frame, wherein the distance is used to reflect the similarity between the simulated key frame and the current key frame; The simulation keyframe whose distance meets the preset condition is regarded as the successfully matched simulation keyframe.
8. The method according to claim 7, characterized in that The step of taking the simulated key frame whose distance meets the preset condition as the successfully matched simulated key frame includes: A simulated key frame with the smallest distance between the descriptor and the current key frame is selected from the multiple simulated key frames as a successfully matched simulated key frame.
9. A movable device, characterized in that: The removable device comprises a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Loopback detection method, point cloud map construction method, electronic equipment and storage medium
CN114187418A
Robot relocalization method and apparatus, and storage medium and electronic device
WO2022012229A1
