Positioning method and system, robot, electronic equipment, medium and program product

Abstract

The invention provides a positioning method and system, a robot, electronic equipment, a medium and a program product. The positioning method comprises the steps of obtaining a plurality of grid semantic maps of a current environment where a target object is located; acquiring a plurality of environment images of the surrounding environment of the target object at the same acquisition moment, and acquiring environment semantic information according to the plurality of environment images; and performing semantic matching on the environment semantic information and the grid semantic map to obtain a matching result, and taking the matching result as target positioning information of the target object. According to the method and the device, high efficiency and high precision of target object positioning are realized by introducing semantic information, the reliability of a positioning result is effectively improved, and particularly, under the condition that visual features are insufficient or interference exists, the problem that a single mode of traditional visual or geometric matching is easy to fail is effectively solved; the situation that the positioning result is low in precision and even fails is avoided, and it is guaranteed that positioning can be conducted timely and reliably in a changeable environment.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a positioning method, system, robot, electronic device, medium and program product. Background Art

[0002] In some application scenarios (such as robot application scenarios), initial positioning is one of the most important links; initial positioning refers to the robot accurately determining its position and posture through the initial positioning function when it is started or the positioning is lost, thereby providing a basic guarantee for subsequent path planning and task execution. However, the current positioning solutions generally have problems such as high cost and insufficient accuracy. Summary of the invention

[0003] The technical problem to be solved by the present disclosure is to overcome the above-mentioned defects in the prior art and to provide a positioning method, system, robot, electronic device, medium and program product.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] The present disclosure provides a positioning method, the positioning method comprising:

[0006] Obtain several grid semantic maps of the current environment where the target object is located;

[0007] Acquire a plurality of environment images of the surrounding environment of the target object at the same acquisition time, and acquire environment semantic information according to the plurality of environment images;

[0008] Semantic matching is performed on the environmental semantic information and the grid semantic map to obtain a matching result, and the matching result is used as the target positioning information of the target object.

[0009] Preferably, the step of obtaining a plurality of grid semantic maps of the current environment where the target object is located includes:

[0010] Get pre-built visual feature maps and vector maps;

[0011] Based on each of the environment images, obtaining a plurality of key frames and frame information of each of the key frames from the visual feature map;

[0012] Segmenting the vector map based on different frame information to obtain a plurality of sub-maps;

[0013] Each of the sub-maps is rasterized to obtain a plurality of raster semantic maps.

[0014] Preferably, the step of acquiring a plurality of key frames from the visual feature map based on each of the environment images comprises:

[0015] Obtaining a first global descriptor corresponding to each of the environment images and a second global descriptor corresponding to each frame of the image in the visual feature map;

[0016] In response to the similarity between the first global descriptor and the second global descriptor satisfying a first preset value, a corresponding frame image is extracted from the visual feature map as the key frame to obtain a plurality of the key frames.

[0017] Preferably, the step of acquiring environmental semantic information based on a plurality of environmental images comprises:

[0018] Extracting the first semantic information corresponding to each of the environmental images, and splicing a plurality of the first semantic information to obtain the environmental semantic information;

[0019] Alternatively, a plurality of the environment images are spliced ​​together to obtain the spliced ​​environment image, and a semantic information extraction process is performed on the spliced ​​environment image to obtain the environment semantic information.

[0020] Preferably, the positioning method further includes:

[0021] Get pre-built visual feature maps;

[0022] Based on each of the environment images, obtaining a plurality of key frames and frame information of each of the key frames from the visual feature map;

[0023] The step of performing semantic matching between the environmental semantic information and the grid semantic map to obtain a matching result includes:

[0024] Using a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of the grid semantic maps, so as to obtain an optimal solution as the matching result;

[0025] Wherein, the optimal solution corresponds to target posture information and / or target position information.

[0026] Preferably, the step of using a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of the grid semantic maps to obtain an optimal solution as the matching result includes:

[0027] Using the branch and bound algorithm, performing branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to one of the key frames to obtain a candidate solution corresponding to the current key frame;

[0028] Wherein, the candidate solutions corresponding to different key frames are obtained by parallel processing based on multi-threading technology;

[0029] From the plurality of candidate solutions corresponding to the plurality of key frames, a candidate solution satisfying a preset condition is selected as the optimal solution.

[0030] Preferably, the step of using the branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to one of the key frames to obtain a candidate solution corresponding to the current key frame includes:

[0031] Based on a plurality of preset resolutions, generating a probability grid map containing occupancy probabilities for each of the grid semantic maps at each of the preset resolutions;

[0032] Converting the environmental semantic information into point cloud information;

[0033] Wherein, the environmental semantic information includes information of different semantic types, and the state values ​​of the point cloud information corresponding to the information of different semantic types are different;

[0034] Using the branch and bound algorithm, based on the point cloud information, the frame information of the key frame, and a plurality of probability grid maps at each of the preset resolutions, an initial solution is obtained by matching the environmental semantic information with each of the grid semantic maps;

[0035] A maximum value is selected from the multiple initial solutions to serve as the candidate solution corresponding to the current key frame.

[0036] Preferably, the step of using the branch and bound algorithm to obtain an initial solution by matching the environmental semantic information with each of the grid semantic maps based on the point cloud information, the frame information of the key frame, and a plurality of the probability grid maps at each of the preset resolutions comprises:

[0037] Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles;

[0038] According to the position information in the frame information of the current key frame, projecting each new point cloud information onto a plurality of probability grid maps at the corresponding preset resolution according to a preset projection rule;

[0039] According to the preset translation rules, each of the point cloud data is translated multiple times in the probability grid map at the current preset resolution, and based on the preset calculation rules, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution.

[0040] Preferably, the preset projection rule includes projecting in order from low to high according to the preset resolution;

[0041] The preset calculation rule includes performing calculations in order from low to high according to the preset resolutions.

[0042] Preferably, the step of calculating different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose based on a preset calculation rule to obtain the highest matching score at the corresponding preset resolution includes:

[0043] Calculate the semantic matching scores of different candidate poses after each translation at the lowest preset resolution to obtain the highest matching score at the lowest preset resolution;

[0044] In response to the highest matching score at the lowest preset resolution being greater than a second preset value, continue to calculate the corresponding highest matching score at a higher level of preset resolution until the highest value of the highest matching scores at all the preset resolutions is obtained as the candidate solution under the current key frame.

[0045] Preferably, the positioning method further includes:

[0046] In response to the highest matching score at the lowest preset resolution being less than the second preset value, it is determined that the current key frame is invalid, and branch and bound processing is stopped on the probability grid maps at other preset resolutions.

[0047] Preferably, the step of selecting candidate solutions that meet preset conditions as the optimal solution includes:

[0048] The screened candidate solutions are verified, and in response to satisfying a verification condition, the candidate solution with the highest matching score is selected as the optimal solution from all the screened candidate solutions satisfying the verification condition.

[0049] Preferably, the positioning method further includes:

[0050] In response to all the candidate solutions not satisfying the verification condition, it is determined that the currently extracted key frames are unusable, and a new number of key frames are re-extracted to re-perform positioning processing.

[0051] Preferably, the verification condition includes that the highest value corresponding to the screened candidate solution is greater than a third preset value, and the scores of different semantic types corresponding to the candidate postures at the corresponding preset resolution are all greater than the third preset value.

[0052] Preferably, the target object includes a robot.

[0053] The present disclosure also provides a positioning system, the positioning system comprising:

[0054] A raster map acquisition module obtains several raster semantic maps of the current environment where the target object is located;

[0055] An image acquisition module, used to acquire a plurality of environment images of the surrounding environment of the target object at the same acquisition moment;

[0056] A semantic information acquisition module, used for acquiring environmental semantic information according to a plurality of environmental images;

[0057] The semantic matching module is used to perform semantic matching on the environmental semantic information and the grid semantic map to obtain a matching result, and use the matching result as the target positioning information of the target object.

[0058] Preferably, the grid map acquisition module includes:

[0059] An initial map acquisition unit, used to acquire a pre-built visual feature map and a vector map;

[0060] A key frame acquisition unit, used for acquiring a plurality of key frames and frame information of each key frame from the visual feature map based on each of the environment images;

[0061] A map segmentation unit, used for segmenting the vector map based on different frame information to obtain a plurality of sub-maps;

[0062] The grid semantic map unit is used to perform grid processing on each of the sub-maps to obtain a plurality of the grid semantic maps.

[0063] Preferably, the key frame acquisition unit includes:

[0064] Obtaining a first global descriptor corresponding to each of the environment images and a second global descriptor corresponding to each frame of the image in the visual feature map;

[0065] In response to the similarity between the first global descriptor and the second global descriptor satisfying a first preset value, a corresponding frame image is extracted from the visual feature map as the key frame to obtain a plurality of the key frames.

[0066] Preferably, the semantic information acquisition module includes:

[0067] A first semantic information extraction unit, used to extract first semantic information corresponding to each of the environment images;

[0068] A semantic information concatenation unit, used for concatenating a plurality of the first semantic information to obtain the environmental semantic information;

[0069] or,

[0070] An image stitching unit, used for stitching a plurality of the environment images to obtain a stitched environment image;

[0071] The environment semantic information extraction unit is used to perform semantic information extraction processing on the spliced ​​environment image to obtain the environment semantic information.

[0072] Preferably, the semantic matching module is further used to use a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of the grid semantic maps, so as to obtain an optimal solution as the matching result;

[0073] Wherein, the optimal solution corresponds to target posture information and / or target position information.

[0074] Preferably, the grid map acquisition module includes:

[0075] An initial map acquisition unit, used to acquire a pre-built visual feature map;

[0076] A key frame acquisition unit, used for acquiring a plurality of key frames and frame information of each key frame from the visual feature map based on each of the environment images;

[0077] The semantic matching module includes:

[0078] A candidate solution acquisition unit, configured to use the branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to one of the key frames, so as to obtain a candidate solution corresponding to the current key frame;

[0079] Wherein, the candidate solutions corresponding to different key frames are obtained by parallel processing based on multi-threading technology;

[0080] The matching result acquisition unit is used to select a candidate solution that meets a preset condition from the candidate solutions corresponding to the key frames to serve as the optimal solution.

[0081] Preferably, the candidate solution acquisition unit is used to:

[0082] A probability grid map generation subunit, for generating a probability grid map containing occupancy probability at each of the grid semantic maps based on a plurality of preset resolutions;

[0083] An information conversion subunit, used for converting the environmental semantic information into point cloud information;

[0084] Wherein, the environmental semantic information includes information of different semantic types, and the state values ​​of the point cloud information corresponding to the information of different semantic types are different;

[0085] An initial solution acquisition subunit, configured to adopt the branch and bound algorithm to obtain an initial solution by matching the environmental semantic information with each of the grid semantic maps based on the point cloud information, the frame information of the key frame, and a plurality of the probability grid maps at each of the preset resolutions;

[0086] The candidate solution acquisition subunit is used to select a maximum value from the multiple initial solutions to serve as the candidate solution corresponding to the current key frame.

[0087] Preferably, the initial solution acquisition subunit is further used for:

[0088] Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles;

[0089] According to the position information in the frame information of the current key frame, projecting each new point cloud information onto a plurality of probability grid maps at the corresponding preset resolution according to a preset projection rule;

[0090] According to the preset translation rules, each of the point cloud data is translated multiple times in the probability grid map at the current preset resolution, and based on the preset calculation rules, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution.

[0091] Preferably, the preset projection rule includes projecting in order from low to high according to the preset resolution;

[0092] The preset calculation rule includes performing calculations in order from low to high according to the preset resolutions.

[0093] Preferably, the initial solution acquisition subunit is further used for:

[0094] Calculate the semantic matching scores of different candidate poses after each translation at the lowest preset resolution to obtain the highest matching score at the lowest preset resolution;

[0095] In response to the highest matching score at the lowest preset resolution being greater than a second preset value, continue to calculate the corresponding highest matching score at a higher level of preset resolution until the highest value of the highest matching scores at all the preset resolutions is obtained as the candidate solution under the current key frame.

[0096] Preferably, the positioning system further comprises:

[0097] The judgment module is used for determining that the current key frame is invalid in response to the highest matching score at the lowest preset resolution being less than the second preset value, and stopping the branch and bound processing on the probability grid maps at other preset resolutions.

[0098] Preferably, the matching result acquisition unit is used to verify the screened candidate solutions, and in response to satisfying a verification condition, select the candidate solution with the highest matching score from all the screened candidate solutions satisfying the verification condition as the optimal solution.

[0099] Preferably, the matching result acquisition unit is used for determining that the currently extracted key frames are unavailable in response to all the candidate solutions failing to satisfy the verification condition, and re-extracting a new number of key frames to re-perform positioning processing.

[0100] Preferably, the verification condition includes that the highest value corresponding to the screened candidate solution is greater than a third preset value, and the scores of different semantic types corresponding to the candidate postures at the corresponding preset resolution are all greater than the third preset value.

[0101] Preferably, the target object includes a robot.

[0102] The present disclosure also provides a robot, comprising the positioning system as described above.

[0103] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor, and the processor implements the above-mentioned positioning method when executing the computer program.

[0104] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the above positioning method when executed by a processor.

[0105] The present disclosure also provides a computer program product, including a computer program, which implements the above positioning method when executed by a processor.

[0106] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0107] The positive and progressive effects of this disclosure are:

[0108] In the present invention, the extraction based on semantic information is less affected by changes in illumination, and the semantic features in the environment are relatively stable. By introducing semantic information, high efficiency and high precision in positioning the target object can be achieved, and the reliability of the positioning results is effectively improved. In particular, when visual features are insufficient or there is interference, the problem that a single mode of traditional vision or geometric matching is prone to failure can be effectively solved, and the occurrence of low-precision or even failure of positioning results can be avoided, ensuring timely and reliable positioning in a changing environment, with the advantages of high precision, low cost, and environmental robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 This is a flowchart of the positioning method of Embodiment 1 of the present disclosure;

[0110] Figure 2 This is a flowchart of a positioning method according to Embodiment 2 of the present disclosure;

[0111] Figure 3 This is a module schematic diagram of a positioning system according to Embodiment 3 of the present disclosure;

[0112] Figure 4 This is a module schematic diagram of a positioning system according to Embodiment 4 of the present disclosure;

[0113] Figure 5 This is a schematic diagram of the structure of an electronic device according to Embodiment 6 of the present disclosure. DETAILED DESCRIPTION

[0114] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0115] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0116] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] In the application scenarios of robots, initial positioning is one of the most important links. Initial positioning refers to the accurate determination of the robot's position and posture through the initial positioning function when the robot is started or the positioning is lost, thus providing a basic guarantee for subsequent path planning and task execution. The robustness of the initial positioning technology directly affects the application effect of the robot in complex environments.

[0118] At present, the initial positioning of robots mainly relies on laser radar and vision methods. Specifically, (1) the laser radar-based method determines the position by scanning the surrounding environment and calculating the matching degree with the reference map. Although this method has high accuracy, the equipment cost is expensive and it is sensitive to environmental factors such as dust and smoke, which limits it in some practical applications. In addition, the performance of laser radar in narrow spaces may also be affected. (2) The visual initial positioning method based on global descriptors relies on extracting and matching the global descriptors of key frames to achieve positioning. In practical applications, this method requires pre-building a database containing the descriptors of each key frame, and the initial positioning is determined by comparing the current frame with the descriptors in the database. However, relying solely on vision is easily affected by factors such as lighting changes, dynamic changes in the environment, and occlusion, resulting in the initial positioning results being not robust.

[0119] In summary, although the existing technology provides a variety of initial positioning solutions, the high cost and environmental dependence of lidar, the illumination sensitivity and matching errors of visual methods, and other problems will directly affect the positioning accuracy and real-time performance of the robot, making it difficult to work reliably in a changing environment.

[0120] Based on the above problems, the present disclosure proposes a new positioning solution, which can effectively solve the problem that the traditional visual or geometric matching single mode is prone to failure when the visual features are insufficient or there is interference, avoid the situation where the positioning result is low in accuracy or even fails, and ensure timely and reliable positioning in a changing environment. It has the advantages of high accuracy, low cost and environmental robustness. Specifically:

[0121] Example 1

[0122] like Figure 1 As shown, the positioning method of this embodiment includes:

[0123] S101, obtaining a plurality of grid semantic maps of the current environment where the target object is located;

[0124] Among them, the target objects include but are not limited to robots;

[0125] For the robot, the current environment (such as a parking lot) includes parking spaces, lane lines, road speed bumps, road arrows, pedestrian zebra crossings, etc.

[0126] The raster semantic map is obtained by rasterizing semantic information including parking spaces, lane lines, road speed bumps, road arrows, pedestrian zebra crossings, etc.

[0127] S102, acquiring a plurality of environment images of the surrounding environment of the target object at the same acquisition time;

[0128] For a robot, multiple environment images taken at the same acquisition time by multiple surround-view cameras installed on the robot, or multiple environment images taken at multiple consecutive acquisition times, can be directly used.

[0129] The multiple surround-view cameras on the robot are directly used for image acquisition, without the need for common and expensive sensors such as LiDAR, ensuring low investment costs.

[0130] S103, obtaining environmental semantic information according to a plurality of environmental images;

[0131] Among them, environmental semantic information includes parking spaces, lane lines, road speed bumps, road arrows, pedestrian zebra crossings and other information.

[0132] S104, semantically matching the environment semantic information with the grid semantic map to obtain a matching result, and using the matching result as target positioning information of the target object.

[0133] For example, the target positioning information is the posture information and position information of the robot. The positioning result obtained by the target positioning information method is used to provide continuous positioning information for subsequent path planning and task execution.

[0134] In this scheme, the extraction based on semantic information is less affected by lighting changes, and the semantic features in the environment are relatively stable. By introducing semantic information, high efficiency and high precision in positioning the target object can be achieved, and the reliability of the positioning results is effectively improved. Especially in the case of insufficient visual features or interference, it can effectively solve the problem that the traditional visual or geometric matching single mode is prone to failure, avoid the occurrence of low accuracy or even failure of positioning results, and ensure timely and reliable positioning in a changing environment. It has the advantages of high precision, low cost and environmental robustness.

[0135] Example 2

[0136] The positioning method of this embodiment is a further improvement on Embodiment 1. Specifically:

[0137] In one feasible solution, Figure 2 As shown, step S101 includes:

[0138] S1011, obtaining a pre-built visual feature map and a vector map;

[0139] Among them, the visual feature map contains information such as key frames of different environments and corresponding global descriptors.

[0140] The vector map contains semantic information such as parking spaces, lane lines, road speed bumps, road arrows, and pedestrian zebra crossings.

[0141] S1012, based on each environment image, obtaining a number of key frames and frame information of each key frame from the visual feature map;

[0142] The frame information includes location information, posture information, and floor information.

[0143] S1013, dividing the vector map based on different frame information to obtain a plurality of sub-maps;

[0144] S1014. Perform rasterization processing on each sub-map to obtain a plurality of raster semantic maps.

[0145] Specifically, according to the floor information, location information, etc. in different frame information, the vector map is segmented to generate a local submap (submap) containing a certain range of the corresponding key frame location, and then the submap is converted into a local raster semantic map through rasterization processing.

[0146] In this solution, based on the visual feature map and the collected environmental image, several key frames are quickly extracted, and then based on the frame information contained in each key frame, the vector image is quickly and effectively segmented to obtain multiple local sub-maps. In this way, non-essential map content in the vector image is removed for subsequent processing operations, thereby ensuring the accuracy of the positioning processing while ensuring a small amount of overall data calculation, thereby achieving efficient and high-precision positioning processing.

[0147] In an practicable solution, the specific steps of acquiring a plurality of key frames from the visual feature map based on each environment image in step S1012 include:

[0148] Obtain a first global descriptor corresponding to each environment image and a second global descriptor corresponding to each frame image in the visual feature map;

[0149] In response to the similarity between the first global descriptor and the second global descriptor satisfying a first preset value, a corresponding frame image is extracted from the visual feature map as a key frame to obtain a plurality of key frames.

[0150] Specifically, after the visual feature map is loaded, the VSLAM (a vision-based simultaneous localization and mapping technology) algorithm is used to search in the visual feature map based on the first global descriptor and the second global descriptor, and the frame images corresponding to the second global descriptor that is very close to each first global descriptor are screened out to obtain several key frames.

[0151] In addition, the number of frame images screened out may be determined according to a preset value, which is determined based on experience and may of course be adjusted according to actual conditions.

[0152] In this solution, the required key frames are extracted from the visual feature map quickly and accurately based on the local descriptor, thereby ensuring the efficiency and accuracy of the overall positioning processing.

[0153] In one feasible solution, step S103 includes:

[0154] Extracting the first semantic information corresponding to each environmental image, and splicing a plurality of first semantic information to obtain environmental semantic information;

[0155] In this scheme, by extracting the semantic information from different images separately and then splicing them, instead of using the semantic information of a single environmental image, the obtained environmental semantic information can more accurately and comprehensively reflect the surrounding environment of the target object. In addition, the extraction of semantic information is less affected by lighting changes and the semantic features in the environment are relatively stable, thereby achieving high efficiency and high precision in positioning the target object and effectively improving the reliability of the positioning results.

[0156] In one feasible solution, step S103 includes:

[0157] A plurality of environmental images are spliced ​​to obtain a spliced ​​environmental image, and a semantic information extraction process is performed on the spliced ​​environmental image to obtain environmental semantic information.

[0158] In this scheme, multiple environmental images captured at the same time are first stitched together, and then semantic information is extracted from the entire stitched image, rather than using the semantic information of a single environmental image. This allows the obtained environmental semantic information to more accurately and comprehensively reflect the environment surrounding the target object, and the extraction of semantic information is less affected by changes in illumination, as well as by the relative stability of semantic features in the environment, thereby achieving efficient and high-precision positioning of the target object and effectively improving the reliability of the positioning results.

[0159] In one feasible solution, step S104 includes:

[0160] S1041, using a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of grid semantic maps, so as to obtain an optimal solution as a matching result;

[0161] Among them, the optimal solution corresponds to the target posture information and / or the target position information.

[0162] Specifically, the core idea of ​​the branch and bound algorithm is to use recursive search to partition the space and calculate the upper or lower bound (bound) for each partition area; if the boundary value of a region indicates that it is impossible to find a better solution, the region and its subregions will be pruned, thereby narrowing the search range and saving computing resources.

[0163] In this solution, the branch and bound algorithm is used for matching, which eliminates the need for precise point cloud matching every time, effectively improving the computational efficiency and thus improving the overall positioning processing accuracy and efficiency.

[0164] In one feasible solution, step S1041 includes:

[0165] S10411, using a branch and bound algorithm, performing branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to a key frame to obtain a candidate solution corresponding to the current key frame;

[0166] Among them, candidate solutions corresponding to different key frames are obtained through parallel processing based on multi-threading technology;

[0167] S10412. Select a candidate solution that meets a preset condition from among a plurality of candidate solutions corresponding to a plurality of key frames as an optimal solution.

[0168] In this solution, multiple key frames are introduced, and multiple possible results are obtained by calculating the candidate solutions under each key frame. Then, the final optimal solution is obtained based on the multiple candidate solutions, avoiding the occurrence of mismatches that may be caused by traditional single matching results, thereby effectively ensuring the accuracy and reliability of target object positioning.

[0169] In addition, the calculation process of each candidate solution corresponds to an independent thread for branch and bound matching, and the calculation process of different candidate solutions adopts multi-threading technology for parallel processing, which can significantly shorten the overall processing time and further improve the overall positioning processing efficiency.

[0170] In one feasible solution, step S10411 includes:

[0171] Based on a number of preset resolutions, generating a probability grid map containing occupancy probability at each preset resolution for each grid semantic map;

[0172] For example, several preset resolutions include 1, 4, 16, 64, and other resolutions;

[0173] Specifically, these probability grid maps at different resolution scales are used as the branch-and-bound layer in the branch-and-bound algorithm. Through these probability grid maps at different resolution scales, the branch-and-bound algorithm can quickly find the upper bound value at different resolution scales, that is, the highest possible matching score of the solution under the node, thereby speeding up the search process and ensuring the high efficiency of the overall positioning processing. Convert environmental semantic information into point cloud information;

[0174] Among them, the environmental semantic information includes information of different semantic types, and the state values ​​of the point cloud information corresponding to the information of different semantic types are different, so as to ensure that the information of different semantic types is clearly distinguished in the point cloud information;

[0175] In addition, in order to ensure the quality and processing efficiency of point cloud information, the Adaptive Voxel Filter is introduced in the data conversion process; when the number of point clouds corresponding to the point cloud information exceeds the set threshold (such as 1500 points), the adaptive voxel filter is used to downsample the point cloud information; wherein, the adaptive voxel filter filters out redundant data points and retains representative data points based on the size of the adaptive voxels and the point cloud density, so as to ensure that the point cloud information can be processed and transmitted more efficiently while meeting the accuracy requirements.

[0176] The branch and bound algorithm is used to obtain the initial solution by matching the environment semantic information with each grid semantic map based on the point cloud information, the frame information of the key frame, and several probability grid maps at each preset resolution;

[0177] The maximum value is selected from multiple initial solutions as the candidate solution corresponding to the current key frame.

[0178] In this scheme, the position information, posture information in the frame information of each key frame, the probability grid map at each preset resolution, and the spliced ​​environmental semantic information are encapsulated into a data packet and sent to the branch and bound algorithm for semantic matching processing to obtain the candidate solutions under the processing thread corresponding to each key frame, which ensures the data processing efficiency while also ensuring the reliability of the positioning processing results.

[0179] In an practicable solution, a branch and bound algorithm is used to obtain an initial solution by matching the environment semantic information with each grid semantic map based on the point cloud information, the frame information of the key frame, and a plurality of probability grid maps at each preset resolution, including:

[0180] Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles;

[0181] Among them, considering that there may be certain errors in the position information after the initial rotation of the point cloud information, the point cloud information is rotated multiple times to obtain a rotated point cloud data set; for example, N rotations are performed in sequence in the clockwise direction, and each rotation angle is A°; and N rotations are performed in sequence in the counterclockwise direction, and each rotation angle is A°; wherein the values ​​of N and M are determined by pre-setting, and of course can also be adjusted according to the actual scenario.

[0182] According to the position information in the frame information of the current key frame, each new point cloud information is projected onto a number of probability grid maps at a corresponding preset resolution according to a preset projection rule;

[0183] According to the preset translation rules, each point cloud data is translated multiple times in the probability grid map at the current preset resolution, and based on the preset calculation rules, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution.

[0184] The preset translation rule includes setting the translation direction and the translation distance in the corresponding direction; in addition, for probability grid maps with different preset resolutions, the preset translation rules for the point cloud information therein may be different, and matching settings are required to further ensure the accuracy of the final matching determination. The preset translation rules under the corresponding preset resolution are determined by presetting, and of course they can also be adjusted according to the actual scene.

[0185] In this scheme, by rotating, translating, projecting and other processing on the point cloud data corresponding to the environmental semantic information, different candidate poses in the probability grid map at a preset resolution are obtained, and then the candidate pose with the highest matching score at the current preset resolution is obtained; for each preset resolution, a processing method is adopted to obtain the candidate pose with the highest corresponding matching score, so as to determine the final candidate pose as the positioning information of the target object, which is not based on a single preset resolution, but takes into account the matching conditions at other preset resolutions for comprehensive determination, thereby ensuring the accuracy and reliability of the positioning result.

[0186] In an implementable solution, the preset projection rule includes performing projection in sequence from low to high according to the preset resolution; the preset calculation rule includes performing calculation in sequence from low to high according to the preset resolution.

[0187] In this scheme, for the raster semantic maps with different preset resolutions projected with point cloud information, the raster semantic map with the lowest resolution is first calculated and processed (that is, the raster semantic map with the lowest resolution is used as the top layer of branch and bound in the branch and bound algorithm), and the candidate poses corresponding to the point cloud information in the raster semantic map with the lowest resolution translated to different positions and the corresponding semantic matching scores are obtained. Then, the semantic matching scores at each translation position are sorted (such as from high to low) to obtain the highest value among these semantic matching scores, which is used as the highest matching score corresponding to the raster semantic map with the lowest resolution.

[0188] Specifically, the semantic matching scores of different candidate poses in the grid semantic map at any preset resolution can be calculated using the following calculation formula:

[0189]

[0190] Among them, Score represents the semantic matching score corresponding to each candidate pose, valid i is the number of points in the point cloud data of the raster semantic map at the current preset resolution that hits the i-th semantic area, total i is the total number of semantic points of the i-th type participating in the matching in the point cloud data in the grid semantic map at the current preset resolution, N represents the number of different semantic types, ω i Represents the preset weight value, i is a positive integer.

[0191] In an implementable solution, based on a preset calculation rule, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution, including:

[0192] Calculate the semantic matching scores of different candidate poses after each translation at the lowest preset resolution to obtain the highest matching score at the lowest preset resolution;

[0193] In response to the highest matching score at the lowest preset resolution being greater than the second preset value, continue to calculate the corresponding highest matching score at a higher preset resolution until the highest value among the highest matching scores at all preset resolutions is obtained as a candidate solution for the current key frame.

[0194] After obtaining the candidate pose with the highest matching score corresponding to the raster semantic map at the lowest preset resolution, determine whether the highest matching score corresponding to the top layer is greater than a set value (such as 0.6). If so, continue the branch and bound processing of the raster semantic map at a higher preset resolution, and so on. Once the highest matching score of the branch and bound processing of the raster semantic maps at other preset resolutions is calculated to be less than the highest matching score corresponding to the top layer, the highest matching score corresponding to the corresponding layer is discarded, and the branch is no longer continued, thereby narrowing the search range. Until the matching result of the branch and bound processing of the raster semantic map at the highest preset resolution is obtained, it is continuously updated through iteration to screen out the highest value among the highest matching scores at all preset resolutions, and the highest value is the candidate solution corresponding to the current key frame.

[0195] In this scheme, when it is determined that the highest matching score corresponding to the top layer is greater than the set value, branch and bound processing is performed on the probability grid maps at other preset resolutions to avoid the situation where the highest matching score corresponding to the top layer is less than the set value, indicating that the current key frame is invalid and there is no need to continue data analysis. In this way, branch and bound processing is still performed on the probability grid maps at all preset resolutions, thereby effectively avoiding unnecessary waste of data computing resources.

[0196] In an implementable solution, the positioning method further includes:

[0197] In response to the highest matching score at the lowest preset resolution being less than a second preset value, it is determined that the current key frame is invalid, and branch and bound processing on probability grid maps at other preset resolutions is stopped.

[0198] In this scheme, when the highest matching score corresponding to the top layer is less than the set value, it is promptly analyzed that the current key frame is invalid, and the probability grid map under all preset resolutions no longer needs to be processed by branch and bound, that is, the branch and bound matching processing of the independent thread corresponding to the current key frame is stopped, effectively avoiding unnecessary waste of data computing resources.

[0199] In an implementable solution, the step of selecting candidate solutions that meet preset conditions as the optimal solution includes:

[0200] The screened candidate solutions are verified, and in response to satisfying the verification condition, a candidate solution with the highest matching score is selected as the optimal solution from all the screened candidate solutions that satisfy the verification condition.

[0201] In this scheme, the candidate solutions corresponding to each key frame need to be further verified, that is, the multiple candidate solutions that determine the final optimal solution are further constrained and screened, so as to select the candidate solution with the highest matching score among all the candidate solutions that pass the test conditions as the final optimal solution. The optimal solution is the positioning information of the target object, including posture information and position information, that is, several candidate solutions branched and bounded by different key frames and further verified. Through a variety of guarantee mechanisms, the occurrence of mismatching is effectively avoided and the accuracy of the positioning results is guaranteed.

[0202] In an implementable solution, the positioning method further includes:

[0203] In response to all candidate solutions not satisfying the verification condition, it is determined that the currently extracted key frames are unusable, and new key frames are re-extracted to re-perform the positioning process.

[0204] In this scheme, if the candidate solutions under each key frame cannot meet the further verification conditions, it means that the multiple key frames currently extracted are not suitable. Therefore, it is necessary to re-match based on the global descriptors in the image and the global descriptors of different frames in the visual feature map to obtain a new batch of key frames, and then recalculate them so that the next round of positioning processing can be carried out in time, efficient processing of abnormal situations can be achieved, and the smoothness and reliability of the entire positioning processing process can be ensured.

[0205] In an implementable solution, the verification condition includes that the highest value corresponding to the screened candidate solutions is greater than a third preset value, and the scores of different semantic types corresponding to the candidate poses at the corresponding preset resolution are all greater than the third preset value.

[0206] In this scheme, the highest value of the candidate solution is set to be greater than the set value, and the scores of at least two semantic types in the point cloud information are also set to be greater than the set value. Only when both conditions are met at the same time, the candidate solution is determined to have passed the verification, so as to effectively ensure the quality of the candidate solutions at the screening point, so as to ensure the accuracy and reliability of the determination of the most likely solution.

[0207] The following takes the target object as a robot as an example to specifically explain the working principle of the positioning solution of this embodiment:

[0208] (1) The robot is equipped with multiple (e.g., 4) surround view cameras, and the corresponding environmental images at the same acquisition time are obtained based on these surround view cameras;

[0209] (2) Using deep learning technology to extract semantic information from these environmental images to obtain semantic information such as parking spaces, lane lines, road speed bumps, road arrows, and pedestrian zebra crossings;

[0210] Splicing semantic information of multiple frames corresponding to multiple frames of environmental images to obtain environmental semantic information;

[0211] Specifically, the semantic information of multiple frames can be spliced ​​using DR (dead reckoning) information. DR is the odometer data obtained by dead reckoning of the robot's wheel speed pulses combined with the inertial measurement unit (IMU).

[0212] (4) Loading the pre-built visual feature map and vector map; after loading the map, using the VSLAM algorithm to filter out multiple key frames from the visual feature map based on the global descriptor corresponding to each frame in the visual feature map and the global descriptor corresponding to each environmental image; extracting the posture information, floor information, location information, etc. in each key frame;

[0213] Based on the floor information and location information in each key frame, the vector map is segmented to obtain a number of local sub-maps within a certain range of the key frame location, and these sub-maps are rasterized to obtain a number of raster semantic maps.

[0214] (5) Using a branch and bound algorithm, the environment semantic information obtained in step (2) is matched with the plurality of grid semantic maps obtained in step (4) to obtain a matching result, which is the location of the robot (or initial positioning), including the location information and posture information of the robot; specifically:

[0215] (51) Based on different preset resolutions (such as 1, 4, 16, 64, etc.), each grid semantic map is processed to generate a probability grid map containing occupancy probability at each preset resolution (as a layer in branch and bound);

[0216] (52) It is necessary to convert the environmental semantic information obtained by the splicing process into point cloud information. During the conversion, different state values ​​will be assigned to the point cloud information according to the semantic type to ensure that data of each semantic type are distinguished in the point cloud information. When the number of point clouds in the point cloud information exceeds the set threshold (such as 1500 points), the point cloud information will be downsampled using an adaptive voxel filter. The filter will automatically filter out redundant data points based on the size of the adaptive voxels and the point cloud density, and retain representative data points, ensuring that the point cloud information can be processed and transmitted more efficiently while meeting the accuracy requirements.

[0217] Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles;

[0218] According to the position information in the frame information of the current key frame, each new point cloud information is projected onto several probability grid maps at each preset resolution;

[0219] (53) Starting from the probability grid map corresponding to the lowest preset resolution (the top layer in branch and bound), each point cloud data is translated multiple times in the probability grid map at the current preset resolution according to the preset translation rule, and based on the preset calculation rule, different candidate poses after each translation at the lowest preset resolution are calculated, as well as the semantic matching score corresponding to each candidate pose, so as to obtain the highest matching score at each preset resolution;

[0220] (54) In response to the highest matching score at the lowest preset resolution being greater than a second preset value, continue to calculate the corresponding highest matching score at a higher preset resolution; once the highest matching score of the branch and bound processing of the grid semantic map at other preset resolutions is calculated to be less than the highest matching score corresponding to the top layer, discard the highest matching score corresponding to the corresponding layer and stop branching, thereby narrowing the search range; until the matching result of the branch and bound processing of the grid semantic map at the highest preset resolution is obtained, it is continuously updated through iteration to screen out the highest value among the highest matching scores at all preset resolutions, and the highest value is the candidate solution corresponding to the current key frame;

[0221] In which, in response to the highest matching score at the lowest preset resolution being less than a second preset value, determining that the current key frame image key frame is invalid, and stopping the branch and bound processing on the probability grid maps at other preset resolutions;

[0222] (55) For each candidate solution corresponding to each key frame, a verification step is set, such as setting the highest value of the candidate solution to be greater than a set value, and the scores of at least two semantic types in the point cloud information also need to be greater than the set value. Only when both conditions are met at the same time, the candidate solution is determined to have passed the verification;

[0223] (56) From all the candidate solutions that meet the verification conditions, the candidate solution with the highest matching score is selected as the optimal solution and used as the target position information of the robot.

[0224] Example 3

[0225] like Figure 3 As shown, the positioning system in this embodiment includes:

[0226] The grid map acquisition module 1 acquires several grid semantic maps of the current environment where the target object is located;

[0227] Image acquisition module 2, used to acquire a plurality of environment images of the surrounding environment of the target object at the same acquisition moment;

[0228] A semantic information acquisition module 3, used to acquire environmental semantic information based on a plurality of environmental images;

[0229] The semantic matching module 4 is used to perform semantic matching between the environmental semantic information and the grid semantic map to obtain a matching result, and use the matching result as the target positioning information of the target object.

[0230] In this scheme, the extraction based on semantic information is less affected by lighting changes, and the semantic features in the environment are relatively stable. By introducing semantic information, high efficiency and high precision in positioning the target object can be achieved, and the reliability of the positioning results is effectively improved. Especially in the case of insufficient visual features or interference, it can effectively solve the problem that the traditional visual or geometric matching single mode is prone to failure, avoid the occurrence of low accuracy or even failure of positioning results, and ensure timely and reliable positioning in a changing environment. It has the advantages of high precision, low cost and environmental robustness.

[0231] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.

[0232] Example 4

[0233] In one feasible solution, Figure 4 As shown, the grid map acquisition module 1 includes:

[0234] An initial map acquisition unit 5, used to acquire a pre-built visual feature map and a vector map;

[0235] A key frame acquisition unit 6, used for acquiring a number of key frames and frame information of each key frame from the visual feature map based on each environment image;

[0236] A map segmentation unit 7, used for segmenting the vector map based on different frame information to obtain a plurality of sub-maps;

[0237] The grid semantic map unit 8 is used to perform grid processing on each sub-map to obtain a plurality of grid semantic maps.

[0238] In an implementable solution, the key frame acquisition unit 6 is used to:

[0239] Obtain a first global descriptor corresponding to each environment image and a second global descriptor corresponding to each frame image in the visual feature map;

[0240] In response to the similarity between the first global descriptor and the second global descriptor satisfying a first preset value, a corresponding frame image is extracted from the visual feature map as a key frame to obtain a plurality of key frames.

[0241] In an implementable solution, the semantic information acquisition module 3 includes:

[0242] A first semantic information extraction unit 9, used to extract the first semantic information corresponding to each environment image;

[0243] The semantic information concatenation unit 10 is used to concatenate a plurality of first semantic information to obtain environmental semantic information;

[0244] or,

[0245] An image stitching unit 11 is used to stitch a plurality of environment images to obtain a stitched environment image;

[0246] The environment semantic information extraction unit 12 is used to perform semantic information extraction processing on the spliced ​​environment image to obtain environment semantic information.

[0247] In an practicable solution, the semantic matching module 4 is further used to use a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of grid semantic maps, so as to obtain an optimal solution as a matching result;

[0248] Among them, the optimal solution corresponds to the target posture information and / or the target position information.

[0249] In an implementable solution, the semantic matching module 4 includes:

[0250] The candidate solution acquisition unit 13 is used to use a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to a key frame to obtain a candidate solution corresponding to the current key frame;

[0251] Among them, candidate solutions corresponding to different key frames are obtained through parallel processing based on multi-threading technology;

[0252] The matching result acquisition unit 14 is used to select a candidate solution that meets a preset condition from a plurality of candidate solutions corresponding to a plurality of key frames as an optimal solution.

[0253] In an implementable solution, the candidate solution obtaining unit 13 is used to:

[0254] A probability grid map generation subunit, for generating a probability grid map containing occupancy probability at each preset resolution for each grid semantic map based on a number of preset resolutions;

[0255] An information conversion subunit, used to convert environmental semantic information into point cloud information;

[0256] Among them, the environmental semantic information includes information of different semantic types, and the state values ​​of the point cloud information corresponding to the information of different semantic types are different;

[0257] An initial solution acquisition subunit is used to adopt a branch and bound algorithm to obtain an initial solution by matching the environment semantic information with each grid semantic map based on the point cloud information, the frame information of the key frame, and a number of probability grid maps at each preset resolution;

[0258] The candidate solution obtaining subunit is used to select the maximum value from multiple initial solutions as the candidate solution corresponding to the current key frame.

[0259] In an implementable solution, the initial solution acquisition subunit is further used for:

[0260] Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles;

[0261] According to the position information in the frame information of the current key frame, each new point cloud information is projected onto a number of probability grid maps at a corresponding preset resolution according to a preset projection rule;

[0262] According to the preset translation rules, each point cloud data is translated multiple times in the probability grid map at the current preset resolution, and based on the preset calculation rules, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution.

[0263] In an implementable solution, the preset projection rule includes projecting in order from low to high according to preset resolutions;

[0264] The preset calculation rule includes calculating in order from low to high according to the preset resolution.

[0265] In an implementable solution, the initial solution acquisition subunit is further used for:

[0266] Calculate the semantic matching scores of different candidate poses after each translation at the lowest preset resolution to obtain the highest matching score at the lowest preset resolution;

[0267] In response to the highest matching score at the lowest preset resolution being greater than the second preset value, continue to calculate the corresponding highest matching score at a higher preset resolution until the highest value among the highest matching scores at all preset resolutions is obtained as a candidate solution for the current key frame.

[0268] In an practicable solution, the positioning system further includes:

[0269] The judgment module 15 is used to determine that the current key frame is invalid in response to the highest matching score at the lowest preset resolution being less than a second preset value, and stop performing branch and bound processing on the probability grid maps at other preset resolutions.

[0270] In an implementable solution, the matching result acquisition unit 14 is used to verify the screened candidate solutions, and in response to satisfying the verification condition, select the candidate solution with the highest matching score as the optimal solution from all the screened candidate solutions satisfying the verification condition.

[0271] In an implementable solution, the matching result acquisition unit 14 is used to determine that the currently extracted key frames are unusable in response to all candidate solutions not satisfying the verification condition, and re-extract a number of new key frames to re-perform positioning processing.

[0272] In an implementable solution, the verification condition includes that the highest value corresponding to the screened candidate solutions is greater than a third preset value, and the scores of different semantic types corresponding to the candidate poses at the corresponding preset resolution are all greater than the third preset value.

[0273] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, in which the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution.

[0274] Example 5

[0275] The robot in this embodiment includes the positioning system as described above.

[0276] The robot in this embodiment is integrated with the above-mentioned positioning system, which can achieve high efficiency and high precision in positioning the target object, effectively solve the problem that the traditional single mode of vision or geometric matching is prone to failure, avoid the occurrence of low accuracy or even failure of positioning results, and ensure timely and reliable positioning in a changing environment. It has the advantages of high precision, low cost and environmental robustness, thereby effectively improving the overall product performance of the robot.

[0277] Example 6

[0278] Figure 5 This is a structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure, the electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor, and when the processor executes the computer program, the positioning method described in any of the above embodiments is implemented. Figure 5 The electronic device 90 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0279] like Figure 5 As shown, the electronic device 90 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0280] The bus 93 includes a data bus, an address bus, and a control bus.

[0281] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0282] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0283] The processor 91 executes various functional applications and data processing by running the computer program stored in the memory 92, such as the positioning method provided in any of the above embodiments.

[0284] The electronic device 90 may also communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 95. Furthermore, the electronic device 90 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 96. As shown, the network adapter 96 communicates with other modules of the electronic device 90 via a bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0285] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0286] Example 7

[0287] The embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the positioning method provided by any of the above embodiments is implemented.

[0288] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0289] Example 8

[0290] The embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned positioning methods when executed by a processor.

[0291] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0292] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A positioning method, characterized in that: The positioning method comprises: Obtain several grid semantic maps of the current environment where the target object is located; Acquire a plurality of environmental images of the surrounding environment of the target object at the same acquisition time, and acquire environmental semantic information according to the plurality of environmental images; Semantic matching is performed on the environmental semantic information and the grid semantic map to obtain a matching result, and the matching result is used as the target positioning information of the target object.

2. The positioning method according to claim 1, characterized in that: The step of obtaining a plurality of grid semantic maps of the current environment where the target object is located includes: Get pre-built visual feature maps and vector maps; Based on each of the environment images, obtaining a plurality of key frames and frame information of each of the key frames from the visual feature map; Segmenting the vector map based on different frame information to obtain a plurality of sub-maps; Each of the sub-maps is rasterized to obtain a plurality of the raster semantic maps.

3. The positioning method according to claim 2, characterized in that: The step of acquiring a plurality of key frames from the visual feature map based on each of the environment images comprises: Obtaining a first global descriptor corresponding to each of the environment images and a second global descriptor corresponding to each frame of the image in the visual feature map; In response to the similarity between the first global descriptor and the second global descriptor satisfying a first preset value, a corresponding frame image is extracted from the visual feature map as the key frame to obtain a plurality of the key frames.

4. The positioning method according to claim 1, characterized in that: The step of acquiring environmental semantic information according to a plurality of environmental images comprises: Extracting the first semantic information corresponding to each of the environmental images, and splicing a plurality of the first semantic information to obtain the environmental semantic information; Alternatively, a plurality of the environment images are spliced ​​together to obtain the spliced ​​environment image, and a semantic information extraction process is performed on the spliced ​​environment image to obtain the environment semantic information.

5. The positioning method according to claim 1, characterized in that: The step of performing semantic matching between the environmental semantic information and the grid semantic map to obtain a matching result includes: Using a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of the grid semantic maps, so as to obtain an optimal solution as the matching result; Wherein, the optimal solution corresponds to target posture information and / or target position information.

6. The positioning method according to claim 5, characterized in that: The positioning method further includes: Get pre-built visual feature maps; Based on each of the environment images, obtaining a plurality of key frames and frame information of each of the key frames from the visual feature map; The step of using a branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and a plurality of the grid semantic maps to obtain an optimal solution as the matching result includes: Using the branch and bound algorithm, performing branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to one of the key frames to obtain a candidate solution corresponding to the current key frame; Wherein, the candidate solutions corresponding to different key frames are obtained by parallel processing based on multi-threading technology; From the plurality of candidate solutions corresponding to the plurality of key frames, a candidate solution satisfying a preset condition is selected as the optimal solution.

7. The positioning method according to claim 6, characterized in that: The step of using the branch and bound algorithm to perform branch and bound matching processing on the environmental semantic information and the grid semantic map corresponding to one of the key frames to obtain a candidate solution corresponding to the current key frame includes: Based on a plurality of preset resolutions, generating a probability grid map containing occupancy probabilities for each of the grid semantic maps at each of the preset resolutions; Converting the environmental semantic information into point cloud information; Wherein, the environmental semantic information includes information of different semantic types, and the state values ​​of the point cloud information corresponding to the information of different semantic types are different; Using the branch and bound algorithm, based on the point cloud information, the frame information of the key frame, and a plurality of probability grid maps at each of the preset resolutions, an initial solution is obtained by matching the environmental semantic information with each of the grid semantic maps; A maximum value is selected from the multiple initial solutions to serve as the candidate solution corresponding to the current key frame.

8. The positioning method according to claim 7, characterized in that: The step of using the branch and bound algorithm to obtain an initial solution by matching the environmental semantic information with each of the grid semantic maps based on the point cloud information, the frame information of the key frame, and a plurality of the probability grid maps at each of the preset resolutions comprises: Based on the posture information in the frame information of the current key frame, the point cloud information is rotated to obtain new point cloud information at multiple preset rotation angles; According to the position information in the frame information of the current key frame, projecting each new point cloud information onto a plurality of probability grid maps at the corresponding preset resolution according to a preset projection rule; According to the preset translation rules, each of the point cloud data is translated multiple times in the probability grid map at the current preset resolution, and based on the preset calculation rules, different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose are calculated to obtain the highest matching score at the corresponding preset resolution.

9. The positioning method according to claim 8, characterized in that: The preset projection rule includes projecting in order from low to high according to the preset resolution; The preset calculation rule includes performing calculations in order from low to high according to the preset resolutions.

10. The positioning method according to claim 9, characterized in that: The step of calculating different candidate poses after each translation at the current preset resolution and the semantic matching score corresponding to each candidate pose based on the preset calculation rule to obtain the highest matching score at the preset resolution includes: Calculate the semantic matching scores of different candidate poses after each translation at the lowest preset resolution to obtain the highest matching score at the lowest preset resolution; In response to the highest matching score at the lowest preset resolution being greater than a second preset value, continue to calculate the corresponding highest matching score at a higher level of preset resolution until the highest value of the highest matching scores at all the preset resolutions is obtained as the candidate solution under the current key frame.

11. The positioning method according to any one of claims 8 to 10, characterized in that: The positioning method further includes: In response to the highest matching score at the lowest preset resolution being less than a second preset value, it is determined that the current key frame is invalid, and branch and bound processing is stopped for the probability grid maps at other preset resolutions.

12. The positioning method according to any one of claims 8 to 10, characterized in that: The step of selecting candidate solutions that meet preset conditions as the optimal solution includes: The screened candidate solutions are verified, and in response to satisfying a verification condition, the candidate solution with the highest matching score is selected as the optimal solution from all the screened candidate solutions satisfying the verification condition.

13. The positioning method according to any one of claims 6 to 10, characterized in that: The positioning method further includes: In response to all the candidate solutions not satisfying the verification condition, it is determined that the currently extracted key frames are unusable, and a new number of key frames are re-extracted to re-perform positioning processing.

14. The positioning method according to claim 12, characterized in that: The verification condition includes that the highest value corresponding to the screened candidate solution is greater than a third preset value, and the scores of different semantic types corresponding to the candidate postures at the corresponding preset resolution are all greater than the third preset value.

15. The positioning method according to any one of claims 1 to 10, characterized in that: The target object includes a robot.

16. A positioning system, characterized in that: The positioning system comprises: A raster map acquisition module obtains several raster semantic maps of the current environment where the target object is located; An image acquisition module, used to acquire a plurality of environment images of the surrounding environment of the target object at the same acquisition moment; A semantic information acquisition module, used for acquiring environmental semantic information according to a plurality of environmental images; The semantic matching module is used to perform semantic matching on the environmental semantic information and the grid semantic map to obtain a matching result, and use the matching result as the target positioning information of the target object.

17. A robot, characterized in that: The robot comprises the positioning system of claim 16.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the positioning method according to any one of claims 1 to 15 is implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the positioning method according to any one of claims 1 to 15 is implemented.

20. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the positioning method according to any one of claims 1 to 15 is implemented.

Images