A loopback detection method and device, electronic equipment and storage medium

Abstract

The application provides a loop detection method and device, electronic equipment and storage medium, and relates to the field of automatic driving.The method comprises the following steps: obtaining an initial to-be-matched semantic point cloud and a plurality of local grid subgraphs of multiple resolution levels; performing a preset number of rotation operations and translation operations on the initial to-be-matched semantic point cloud; performing occupancy state matching on each frame of to-be-matched semantic point cloud based on the grid map of the multiple resolution levels corresponding to the current local grid subgraph to obtain a matching result; ending the occupancy state matching in the case that the matching result indicates that the target to-be-matched semantic point cloud exists; and updating the global semantic map based on the target to-be-matched semantic point cloud and the current local grid subgraph at the end of the occupancy state matching.The application solves the problem of large-angle deflection of the to-be-matched semantic point cloud, makes the result of loop detection more accurate, accelerates the matching speed, and makes the loop matching algorithm achieve real-time performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, in particular to a loop detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the field of automatic driving, real-time positioning and mapping are an indispensable part of an autonomous vehicle, which can help the autonomous vehicle to pre-sense the complex information of the road and provide real-time high-precision positioning information, combined with intelligent path planning, so that the vehicle can make correct decisions. In the mapping phase, the autonomous vehicle cannot accurately observe the global information, and it is inevitable to introduce mapping cumulative error. In order to eliminate the cumulative error, the vehicle usually drives for a period of time and then re-drives to the area where the map has been constructed, and performs loop detection to eliminate the cumulative error. In the prior art, point cloud and map matching are the core of loop detection, but most loop detection methods have the problems of large detection error and long matching search time. SUMMARY

[0003] To solve the above technical problems, the present application provides a loop detection method, device, electronic equipment and storage medium, and the scheme is as follows:

[0004] In a first aspect, a loop detection method is provided, comprising:

[0005] obtaining an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid sub-maps, and each local grid sub-map is a multi-resolution level grid map;

[0006] selecting a current local grid sub-map from the plurality of local grid sub-maps; the current local grid sub-map is any local grid sub-map that has not been matched in the plurality of local grid sub-maps;

[0007] performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid sub-map;

[0008] performing occupancy state matching on each frame of semantic point cloud to be matched in the set of semantic point clouds to be matched based on the multi-resolution level grid map corresponding to the current local grid sub-map, to obtain a matching result;

[0009] in the case where the matching result indicates that there is no target semantic point cloud to be matched, performing the step of selecting a current local grid sub-map from the plurality of local grid sub-maps until the matching result indicates that there is the target semantic point cloud to be matched, and ending the occupancy state matching;

[0010] updating the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph at the end of the occupancy state matching.

[0011] Optionally, the method further comprises:

[0012] obtaining an initial global semantic map, and dividing the initial global semantic map into a plurality of local subgraphs;

[0013] performing grid division on each local subgraph according to a first preset resolution to generate a first-level grid map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local subgraph;

[0014] performing a preset number of resolution reduction processes on the first-level grid map corresponding to each local subgraph based on a preset resolution level to obtain each local grid subgraph.

[0015] Optionally, after the current local grid subgraph is selected from the plurality of local grid subgraphs, the method further comprises:

[0016] determining a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

[0017] rotating the first rotation histogram according to a preset unit rotation angle for a plurality of times to obtain a first rotation histogram corresponding to each rotation angle;

[0018] respectively determining a correlation score of the first rotation histogram corresponding to each rotation angle and the second rotation histogram;

[0019] determining a target first rotation histogram corresponding to the highest correlation score, and taking a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

[0020] Optionally, the obtaining a set of semantic point clouds to be matched by performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched comprises:

[0021] determining a rotation range of the rotation operation based on the target rotation angle, and determining a preset number of rotations based on the rotation range;

[0022] obtaining a first geometric center of the initial semantic point cloud to be matched and a second geometric center of the current local grid subgraph;

[0023] determining a translation range of the translation operation based on the first geometric center and the second geometric center, and determining a preset number of translations based on the translation range;

[0024] The initial semantic point cloud to be matched is subjected to a preset number of rotation operations, and a preset number of translation operations are performed at each rotation operation, to obtain the set of semantic point clouds to be matched.

[0025] Optionally, the matching result is obtained based on the occupancy state matching of each frame of semantic point cloud to be matched in the set of semantic point clouds to be matched and the grid map corresponding to the current local grid subgraph at a multi-resolution level.

[0026] The occupancy state matching is performed between each frame of semantic point cloud to be matched in the set of semantic point clouds to be matched and the lowest level grid map corresponding to the current local grid subgraph, to determine a first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map.

[0027] The matching result is obtained based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map.

[0028] Optionally, the matching result is obtained based on the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map.

[0029] The highest first matching degree score is determined from the first matching degree score corresponding to each frame of semantic point cloud to be matched and the lowest level grid map.

[0030] When the highest first matching degree score is less than a preset score threshold, the matching result is determined as a first matching result, and the first matching result indicates that there is no target semantic point cloud to be matched.

[0031] When the highest first matching degree score is not less than the preset score threshold, the occupancy state matching is performed layer by layer from the lowest level grid map corresponding to the current local grid subgraph to the target semantic point cloud to be matched corresponding to the highest first matching degree score, to obtain a second matching degree score of the target semantic point cloud to be matched corresponding to the highest first matching degree score and each level grid map.

[0032] When the second matching degree score is less than the preset score threshold, the occupancy state matching is stopped, and the matching result is determined as the first matching result.

[0033] When the second matching degree score of the target semantic point cloud to be matched and each level grid map is not less than the preset score threshold, the matching result is determined as a second matching result, and the second matching result indicates that the target semantic point cloud to be matched corresponding to the highest first matching degree score is taken as a target semantic point cloud to be matched.

[0034] Optionally, the updating the global semantic map based on the target semantic point cloud to be matched and the current local network subgraph at the end of the occupancy state matching comprises:

[0035] obtaining a rotation angle and a translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched;

[0036] determining optimized pose information of the target semantic point cloud to be matched based on the rotation angle and the translation distance and the current local grid subgraph at the end of the occupancy state matching;

[0037] updating the global semantic map based on the optimized pose information of the target semantic point cloud to be matched.

[0038] Optionally, the method further comprises:

[0039] stitching a surround view image currently collected by a mobile device to obtain an inverse perspective transformation image;

[0040] performing semantic segmentation on the inverse perspective transformation image to obtain a semantic image to be matched;

[0041] performing coordinate mapping on the semantic image to be matched to obtain the initial semantic point cloud to be matched.

[0042] In a second aspect, a loop detection device is provided, comprising:

[0043] an acquisition module configured to acquire an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid subgraphs, and each local grid subgraph is a multi-resolution level grid map;

[0044] a local grid subgraph selection module configured to select a current local grid subgraph from the plurality of local grid subgraphs; the current local grid subgraph is any local grid subgraph that has not been matched in the plurality of local grid subgraphs;

[0045] a semantic point cloud to be matched transformation module configured to perform a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched; a target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

[0046] a matching module configured to perform occupancy state matching on each frame of semantic point cloud to be matched in the set of semantic point clouds to be matched based on a multi-resolution level grid map corresponding to the current local grid subgraph, to obtain a matching result;

[0047] in a case where the matching result indicates that there is no target semantic point cloud to be matched, performing the step of selecting the current local grid subgraph from the plurality of local network subgraphs until the matching result indicates that there is the target semantic point cloud to be matched, and ending the occupancy state matching;

[0048] a map optimization module configured to update the global semantic map based on the target semantic point cloud to be matched and the current local grid subgraph at the end of the occupancy state matching.

[0049] Optionally, the loop detection module further comprises:

[0050] an initial global semantic map acquisition module configured to acquire an initial global semantic map, and divide the initial global semantic map into a plurality of local subgraphs;

[0051] a grid division module configured to perform grid division on each local subgraph according to a first preset resolution, to generate a first-level grid map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local subgraph;

[0052] a local grid subgraph generation module configured to perform preset times of resolution reduction processing on the first-level grid map corresponding to each local subgraph based on a preset resolution level, to obtain each local grid subgraph.

[0053] Optionally, the loop detection module further comprises:

[0054] a rotation histogram determination module configured to determine a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

[0055] a rotation transformation module configured to rotate the first rotation histogram according to a preset unit rotation angle for multiple times, to obtain a first rotation histogram corresponding to each rotation angle;

[0056] a correlation score determination module configured to determine a correlation score of the first rotation histogram corresponding to each rotation angle and the second rotation histogram, respectively;

[0057] a target rotation angle determination module configured to determine a target first rotation histogram corresponding to the highest correlation score, and take a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

[0058] Optionally, the semantic point cloud to be matched transformation module comprises:

[0059] a rotation range determination unit configured to determine a rotation range of the rotation operation based on the target rotation angle, and determine a preset rotation number based on the rotation range;

[0060] a geometric center determination unit, configured to obtain a first geometric center of the initial to-be-matched semantic point cloud and a second geometric center of the current local grid submap;

[0061] a translation range determination unit, configured to determine a translation range of the translation operation based on the first geometric center and the second geometric center, and determine a preset number of translations based on the translation range;

[0062] a to-be-matched semantic point cloud transformation unit, configured to perform a rotation operation for the initial to-be-matched semantic point cloud for a preset number of rotations, and perform a translation operation for each rotation operation for a preset number of translations, to obtain the set of to-be-matched semantic point clouds.

[0063] Optionally, the matching module comprises:

[0064] a first matching degree score determination module, configured to perform occupancy state matching between each frame of to-be-matched semantic point cloud in the set of to-be-matched semantic point clouds and a lowest-level grid map corresponding to the current local grid submap, to determine a first matching degree score corresponding to the each frame of to-be-matched semantic point cloud and the lowest-level grid map.

[0065] a matching result generation module, configured to obtain a matching result based on the first matching degree score corresponding to the each frame of to-be-matched semantic point cloud and the lowest-level grid map.

[0066] Optionally, the matching result generation module comprises:

[0067] a first matching degree score determination unit, configured to determine a highest first matching degree score from the first matching degree scores corresponding to the each frame of to-be-matched semantic point cloud and the lowest-level grid map.

[0068] a first matching result generation unit, configured to determine a matching result as a first matching result when the highest first matching degree score is less than a preset score threshold, the first matching result indicating that there is no target to-be-matched semantic point cloud.

[0069] a second matching degree score determination unit, configured to, when the highest first matching degree score is not less than the preset score threshold, perform occupancy state matching on a to-be-matched semantic point cloud corresponding to the highest first matching degree score starting from the lowest-level grid map corresponding to the current local grid submap, to obtain second matching degree scores of the to-be-matched semantic point cloud corresponding to the highest first matching degree score and each level of grid map; wherein

[0070] The first matching result generation unit is configured to, when the second matching degree score is less than the preset score threshold, stop the state matching, and determine the matching result as the first matching result.

[0071] The target to-be-matched semantic point cloud determination unit is configured to, when the second matching degree scores of the to-be-matched semantic point cloud and each hierarchical grid map are all not less than the preset score threshold, determine the matching result as a second matching result, and the second matching result indicates that the to-be-matched semantic point cloud corresponding to the highest first matching degree score is taken as a target to-be-matched semantic point cloud.

[0072] Optionally, the map optimization module comprises:

[0073] The pose transformation information acquisition unit is configured to acquire a rotation angle and a translation distance of the target to-be-matched semantic point cloud relative to the initial to-be-matched semantic point cloud.

[0074] The optimized pose information determination unit is configured to determine optimized pose information of the target to-be-matched semantic point cloud based on the rotation angle and the translation distance and a current local grid submap at the end of the state matching.

[0075] The map updating unit is configured to update the global semantic map based on the optimized pose information of the target to-be-matched semantic point cloud.

[0076] Optionally, the loop detection device comprises:

[0077] The inverse perspective transformation unit is configured to stitch a loop-around image currently collected by a mobile device to obtain an inverse perspective transformation image.

[0078] The semantic segmentation unit is configured to perform semantic segmentation on the inverse perspective transformation image to obtain a to-be-matched semantic image.

[0079] The coordinate mapping unit is configured to perform coordinate mapping on the to-be-matched semantic image to obtain the initial to-be-matched semantic point cloud.

[0080] In a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the above method.

[0081] In a fourth aspect, a computer readable storage medium is provided, wherein the computer readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the steps of the above method.

[0082] In a fifth aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium, and a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the steps of the above method.

[0083] By using the technical scheme, the present application has the following beneficial effects:

[0084] In the loop detection process, the present application rotates and translates the to-be-matched semantic point cloud, reduces or eliminates the rotation and translation errors of the to-be-matched semantic point cloud and the global semantic map in the matching process, solves the problem of large-angle deflection of the to-be-matched semantic point cloud, makes the loop detection result more accurate, divides the global semantic map into a plurality of resolution hierarchical local grid sub-maps, matches the to-be-matched semantic point cloud and the local grid sub-maps layer by layer in the occupation state, accelerates the matching speed, and makes the loop matching algorithm achieve real-time performance.

[0085] Other features and advantages of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and the same reference numerals generally represent the same components. Other drawings can also be obtained by those skilled in the art without creative labor.

[0087] Figure 1 A flowchart of a loop detection method provided by an embodiment of the present application is shown in the figure.

[0088] Figure 2 An implementation process diagram of an embodiment of the loop detection method provided by the present application is shown in the figure.

[0089] Figure 3 A structure diagram of a loop detection device provided by an embodiment of the present application is shown in the figure.

[0090] Figure 4 A server hardware structure diagram for running the loop detection method provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0091] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0092] The term "one embodiment" or "an embodiment" as used herein means a specific feature, structure, or characteristic that can be included in at least one implementation of the present application. In the description of the present application, it should be understood that the terms "upper", "lower", "top", "bottom", and the like indicate the orientation or positional relationship shown in the drawings based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" can be explicitly or implicitly included one or more features. Moreover, the terms "first", "second" and the like are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0093] The loop detection method provided by the embodiments of the present application can be applied to the instant positioning and mapping stage in autonomous driving, with reference to Figure 1 It is a loop detection method flow diagram provided by an embodiment of the present application. It should be noted that the method operation steps provided in the specification can include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. When the system device or product is executed in practice, it can be executed in sequence or in parallel (for example, parallel processor or multi-threaded processing environment) according to the method order shown in the embodiments or drawings. The loop detection method provided by the embodiments of the present application includes:

[0094] S101, obtaining an initial matching semantic point cloud and a global semantic map; the global semantic map includes a plurality of local grid sub-maps, and each local grid sub-map is a multi-resolution hierarchical grid map;

[0095] In one possible way, the generation method of the initial matching semantic point cloud includes:

[0096] The surround view image currently collected by the mobile device is spliced to obtain an inverse perspective transformation image;

[0097] The inverse perspective transformation image is subjected to semantic segmentation to obtain a to-be-matched semantic image;

[0098] The to-be-matched semantic image is subjected to coordinate mapping to obtain the initial to-be-matched semantic point cloud.

[0099] Specifically, in the embodiment of the present application, the mobile device can be a vehicle, and the image collection unit on the vehicle collects a frame of road image every interval preset time or every interval set distance during the driving of the vehicle. The to-be-matched semantic point cloud is obtained after processing the surround view image collected in the current frame. The surround view image is spliced and subjected to inverse perspective transformation processing to obtain an inverse perspective transformation image. The inverse perspective transformation image is subjected to semantic segmentation to obtain a to-be-matched semantic image. The to-be-matched semantic image can contain lane line information, ground marking information such as zebra crossing, no parking area, and deceleration zone, and is used for matching in the loop detection process. The to-be-matched semantic image is a two-dimensional image, which is subjected to coordinate mapping to a three-dimensional semantic point cloud to obtain an initial to-be-matched semantic point cloud. In another embodiment, the initial to-be-matched semantic point cloud can also be a two-dimensional coordinate of each plane feature point in the map coordinate system obtained by selecting plane feature points in the laser point cloud and according to the pose of the vehicle when the current frame of laser point cloud is collected.

[0100] The global semantic map is a global map relative to the current time obtained by splicing the global map based on the historical frames of images collected by the vehicle. The global semantic map can be a two-dimensional image or a point cloud map. The local grid sub-map is a multi-resolution hierarchical grid map. Each grid is marked with a probability of being occupied by an obstacle using a sliding window algorithm. Here, the obstacle includes lane lines and ground markings such as zebra crossings, no parking areas, and deceleration zones in the local grid sub-map and the to-be-matched semantic point cloud. The grid occupancy probability is used for calculating the matching degree when the to-be-matched semantic point cloud is matched with each local grid sub-map.

[0101] In one possible way, the method for generating each local grid sub-map comprises:

[0102] An initial global semantic map is obtained, and the initial global semantic map is divided into a plurality of local sub-maps;

[0103] Each local sub-map is subjected to grid division according to a first preset resolution to generate a first-level grid map corresponding to each local sub-map. The first preset resolution represents the size of each grid in the first-level grid map corresponding to each local sub-map;

[0104] Each local grid sub-map is obtained by performing a preset number of resolution reduction processes on the first-level grid map corresponding to each local sub-map based on a preset resolution level.

[0105] Specifically, an initial global semantic map that has not been divided is acquired, and the initial global semantic map is divided into a plurality of local sub-maps. The local sub-maps are regarded as feature points, and the entire divided global semantic map can be regarded as a topological map. The local sub-maps are feature points, and the relative position constraint relationship between the local sub-maps is a line. The relative position relationship between the local sub-maps is maintained through a center point, but the relative position error is relatively low between local sub-maps with similar positions. As the number of local sub-maps increases, the movement distance of the robot becomes larger, various random factors, and the like, the error will become larger and larger, so loop detection needs to be performed to correct the error.

[0106] The local sub-maps are grid processed in units of n centimeters, that is, a first preset resolution, to obtain a first-level grid map corresponding to each of the local sub-maps. Since the resolution of each of the first-level grid maps is reduced in the following step, the first-level grid map is the highest-level grid map corresponding to the local sub-map. In the embodiment of the present application, the resolution of each of the first-level grid maps is reduced in units of 2n centimeters, 4n centimeters, and the like. In other embodiments, other preset resolutions can be used to grid divide the local sub-maps to generate a plurality of local grid sub-maps with different resolutions. The larger the grid size, the lower the resolution of the corresponding level grid map, and the fewer the pixel points. For example, when the area of the current local sub-map is 1 square meter and the resolution is 5 cm per pixel, each grid corresponds to 400 pixel points. If the resolution is compressed to 10 cm per pixel, each grid corresponds to 100 pixel points. For the same size of the constructed local sub-map, the pixel points are reduced, and the amount of calculation required for matching is greatly reduced, thereby improving the matching efficiency. It should be noted that each grid of the mth-level grid map represents the grid with the maximum probability in the adjacent 2 m-1

[0107] S102, selecting a current local grid sub-map from a plurality of local grid sub-maps; the current local grid sub-map is any local grid sub-map that has not been matched in the plurality of local grid sub-maps;

[0108] S103, performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched; a target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid sub-map;

[0109] ​Specifically, considering that the to-be-matched semantic point cloud and the local grid subgraph may exist large-angle deflection problems in the matching process, the initial to-be-matched semantic point cloud is subjected to a preset number of rotation operations and translation operations to obtain a to-be-matched semantic point cloud set, which contains multiple frames of to-be-matched semantic point clouds with different rotation angles and different translation amounts.

[0110] In a possible implementation, the step S103 can include the following steps:

[0111] determining a rotation range of the rotation operation based on the target rotation angle, and determining a preset number of rotations based on the rotation range;

[0112] obtaining a first geometric center of the initial to-be-matched semantic point cloud and a second geometric center of the current local grid subgraph;

[0113] determining a translation range of the translation operation based on the first geometric center and the second geometric center, and determining a preset number of translations based on the translation range;

[0114] subjecting the initial to-be-matched semantic point cloud to the preset number of rotation operations, and performing the preset number of translation operations at each rotation operation to obtain the to-be-matched semantic point cloud set.

[0115] Specifically, a target rotation angle, such as 30 degrees, is determined based on a first rotation histogram of the initial to-be-matched semantic point cloud and a second rotation histogram of the current local grid subgraph, and the rotation range of the rotation operation on the initial to-be-matched semantic point cloud is 30 degrees. The rotation number can be set according to actual conditions, such as 20 times, and the rotation angle of the to-be-matched semantic point cloud can be 30-10 degrees to 30+10 degrees, i.e. 20 degrees to 40 degrees.

[0116] Further, the distance between the first geometric center of the initial semantic point cloud to be matched and the second geometric center of the current local grid subgraph is determined, that is, the translation range of the initial semantic point cloud to be matched for the translation operation, and the number of translations can also be set according to actual conditions, for example, the distance is 5 meters, each translation is 0.5 meters, and the total translation is 10 times. It should be noted that the process of the rotation operation and the translation operation in the embodiment of the application is that the translation operation is performed for the preset number of times of translation each time the rotation operation is performed. Taking the above example as a reference, the initial semantic point cloud to be matched is rotated to 20 degrees, and under this rotation condition, the initial semantic point cloud to be matched is translated 10 times, each time by 0.5 meters, to obtain a frame of semantic point cloud to be matched; the initial semantic point cloud to be matched is rotated to 21 degrees, and under this rotation condition, the initial semantic point cloud to be matched is translated 10 times, each time by 0.5 meters, to obtain another frame of semantic point cloud to be matched; and so on, until the rotation is 20 times, to obtain multiple frames of semantic point cloud to be matched with different rotation angles and different translation amounts.

[0117] In a possible implementation, the target rotation angle determination method of the rotation operation includes:

[0118] determining a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid subgraph;

[0119] rotating the first rotation histogram multiple times according to a preset unit rotation angle, to obtain a first rotation histogram corresponding to each rotation angle;

[0120] determining a correlation score of each rotation angle corresponding to the first rotation histogram and the second rotation histogram, respectively;

[0121] determining a target first rotation histogram corresponding to the highest correlation score, and taking the rotation angle corresponding to the target first rotation histogram as the target rotation angle.

[0122] Specifically, refer to Figure 2The first rotation histogram of the initial to-be-matched semantic point cloud is shown in the first rotation histogram example determination process of the initial to-be-matched semantic point cloud. First, the mean value of the coordinates of each point of the initial to-be-matched semantic point cloud is calculated to obtain the centroid c of the initial to-be-matched semantic point cloud. Second, the line connecting each point of the initial to-be-matched semantic point cloud and the centroid is determined, and the angle between each line and the x-axis is calculated. Then, all points are sorted according to the angles. The first point is the point with the smallest angle with the x-axis, and the reference point a is set as the first point. The next point of the reference point a is point b, and the line connecting point b and the reference point a is calculated. The angle between the line and the x-axis is calculated, and the angle is mapped to the first rotation histogram. In the coordinate system of the to-be-matched semantic point cloud, the initial first rotation histogram is divided into several parts according to the preset unit angle. For example, if the unit angle is 2 degrees, the initial first rotation histogram is divided into 180 parts. The degree corresponding to the angle is mapped to the corresponding angle interval of the initial first rotation histogram. The weight of point b is calculated, that is, the perpendicularity of ab and bc is determined. It can be understood that the closer the distance between point a and point b, the closer ab and bc to the vertical, and the greater the weight of point b. The calculation process of other points and point b is consistent. The histogram element in each angle interval of the first rotation histogram is the cumulative value of the weights of the points.

[0123] The current local grid subgraph can contain images collected by multiple frames of historical frames and semantic point clouds obtained by semantic segmentation processing. Similar to the first rotation histogram calculation process of the initial to-be-matched semantic point cloud, the rotation histogram of each frame of semantic point cloud is obtained. The rotation histograms of the semantic point clouds of each frame are superimposed to obtain the second rotation histogram of the current local grid subgraph. Note that this process uses the highest level grid map corresponding to the current local grid subgraph for calculation. Then, the first rotation histogram of the initial to-be-matched semantic point cloud is rotated at different angles. The angle and the number of rotations each time can be different in different embodiments. The correlation between the first rotation histogram at each rotation angle and the second rotation histogram of the current local grid subgraph is calculated to determine the target first rotation histogram corresponding to the highest correlation score. The rotation angle corresponding to the target first rotation histogram is taken as the target rotation angle.

[0124] In step S104, the occupancy state matching of each frame of to-be-matched semantic point cloud in the to-be-matched semantic point cloud set is performed based on the multi-resolution level grid map corresponding to the current local grid subgraph, and a matching result is obtained.

[0125] In one possible implementation, step S104 includes:

[0126] The occupancy state matching of each frame of to-be-matched semantic point cloud in the to-be-matched semantic point cloud set is performed with the lowest level grid map corresponding to the current local grid subgraph, and a first matching degree score corresponding to each frame of to-be-matched semantic point cloud and the lowest level grid map is determined.

[0127] determining a highest first matching degree score from the first matching degree scores corresponding to the lowest level grid map of each frame of the to-be-matched semantic point cloud;

[0128] When the highest first matching degree score is less than a preset score threshold, determining that a matching result is a first matching result, and the first matching result indicates that there is no target to-be-matched semantic point cloud.

[0129] When the highest first matching degree score is not less than the preset score threshold, performing occupancy state matching on the to-be-matched semantic point cloud corresponding to the highest first matching degree score from the lowest level grid map corresponding to the current local grid submap, to obtain second matching degree scores of the to-be-matched semantic point cloud corresponding to the highest first matching degree score and each level grid map; wherein,

[0130] When the second matching degree score is less than the preset score threshold, stopping the occupancy state matching, and determining that the matching result is the first matching result.

[0131] When the second matching degree scores of the to-be-matched semantic point cloud and each level grid map are all not less than the preset score threshold, determining that the matching result is a second matching result, and the second matching result indicates that the to-be-matched semantic point cloud corresponding to the highest first matching degree score is taken as a target to-be-matched semantic point cloud.

[0132] Specifically, the lowest level grid map corresponding to the current local grid submap is a grid map with the lowest resolution, and the occupancy state matching means that each frame of the to-be-matched semantic point cloud is mapped into the lowest level grid map, and each grid occupied by the point cloud is accumulated according to a probability value to calculate a first matching degree score. The greater the probability value accumulation is, the higher the first matching degree score is. The first matching degree scores corresponding to each frame of the to-be-matched semantic point cloud are sorted to determine a to-be-matched semantic point cloud with the highest first matching degree score. Meanwhile, the highest first matching degree score is compared with a preset score threshold. The preset score threshold represents a minimum value of a matching degree score of a to-be-matched semantic point cloud matched with a current level grid map of a current local submap. If the highest first matching degree score is less than the preset score threshold, it indicates that the to-be-matched semantic point cloud fails to match the lowest level grid map of the current local submap, and the occupancy state matching is stopped. Then, other local grid submaps are selected to perform occupancy state matching again.

[0133] If the highest first matching score is not less than the preset score threshold, it is indicated that the matching of the to-be-matched semantic point cloud and the lowest level grid map of the current local subgraph is successful, and the position with the highest score in the lowest level grid map is determined. The to-be-matched semantic point cloud corresponding to the highest first matching score is matched with the current local grid subgraph layer by layer in an occupancy state by using a recursive search strategy, that is, the matching is performed from a low-resolution level to a high-resolution level until the grid map of the highest level is matched.

[0134] In the matching process, if the second matching score of a certain layer is less than the preset score threshold, it is indicated that the matching of the to-be-matched semantic point cloud and the lowest level grid map of the current local subgraph fails, and the occupancy state matching is stopped. Other local grid subgraphs are selected to perform occupancy state matching again until a current local grid subgraph is finally found, the first matching score of the lowest level grid map of the current local grid subgraph and a certain frame of to-be-matched semantic point cloud concentrated in the to-be-matched semantic point cloud is the highest, and the second matching score of the grid map of each level and the to-be-matched semantic point cloud with the highest first matching score is not less than the preset score threshold. The to-be-matched semantic point cloud with the highest first matching score is the target to-be-matched semantic point cloud, and the current local grid subgraph matched successfully is the local grid subgraph of which the loop detection is successful.

[0135] The local grid subgraph is divided into grid maps of multiple resolution levels, and the matching is started from the grid map with the lowest resolution. Since the lower the resolution is, the larger the grid is, and the fewer the pixel points are, the smaller the calculation amount is when the matching score is calculated. Once the matching score is less than the preset score threshold, the matching of the to-be-matched semantic point cloud and the current local grid subgraph is stopped, the matching speed is accelerated, and the real-time performance of the loop matching algorithm is achieved.

[0136] S105, in the case where the matching result indicates that there is no target to-be-matched semantic point cloud, the step of selecting the current local grid subgraph from the multiple local network subgraphs is performed until the matching result indicates that there is the target to-be-matched semantic point cloud, and the occupancy state matching is ended;

[0137] S106, updating the global semantic map based on the target to-be-matched semantic point cloud and the current local grid subgraph at the end of the occupancy state matching.

[0138] In a possible implementation, the step S106 includes:

[0139] The rotation angle and the translation distance of the target to-be-matched semantic point cloud relative to the initial to-be-matched semantic point cloud are obtained.

[0140] determine the optimized pose information of the target semantic point cloud to be matched based on the rotation angle and the translation distance and the current local grid submap at the end of the occupancy state matching;

[0141] update the global semantic map based on the optimized pose information of the target semantic point cloud to be matched.

[0142] Specifically, the pose of the target semantic point cloud to be matched is not necessarily the pose of the initial semantic point cloud to be matched, so the rotation angle and the translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched are determined, and the initial pose of the initial semantic point cloud to be matched relative to the current local grid submap that matches successfully. The rotation angle and the translation distance plus the initial pose are the optimized pose of the target semantic point cloud to be matched. The optimized pose is updated to the global semantic map, and the loop closure detection is completed.

[0143] Corresponding to the loop closure detection method described above, the embodiment of the application also provides a loop closure detection device. Since the loop closure detection device provided by the embodiment of the application corresponds to the loop closure detection method provided by the above several embodiments, the implementation of the loop closure detection method described above is also applicable to the loop closure detection device provided by the embodiment of the application. In the embodiment of the application, it will not be described again.

[0144] Reference Figure 3 It is a loop closure detection device provided by an embodiment of the application. The device has the function of implementing the loop closure detection method in the above method embodiments. The function can be realized by hardware, or the corresponding software can be executed by hardware. The device can include:

[0145] The acquisition module 310 is configured to acquire an initial semantic point cloud to be matched and a global semantic map. The global semantic map includes a plurality of local grid submaps, and each local grid submap is a multi-resolution hierarchical grid map.

[0146] The local grid submap selection module 320 is configured to select a current local grid submap from the plurality of local grid submaps. The current local grid submap is any local grid submap that has not been matched in the plurality of local grid submaps.

[0147] The semantic point cloud to be matched transformation module 330 is configured to perform a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched. The target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid submap.

[0148] The matching module 340 is configured to perform occupancy state matching on each frame of the to-be-matched semantic point cloud in the to-be-matched semantic point cloud set based on the grid map of the multi-resolution level corresponding to the current local grid subgraph, to obtain a matching result.

[0149] In a case where the matching result indicates that there is no target to-be-matched semantic point cloud, the step of selecting the current local grid subgraph from the plurality of local network subgraphs is performed until the matching result indicates that there is the target to-be-matched semantic point cloud, and the occupancy state matching is ended.

[0150] The map optimization module 350 is configured to update the global semantic map based on the target to-be-matched semantic point cloud and the current local grid subgraph at the end of the occupancy state matching.

[0151] Optionally, the loop detection module further comprises:

[0152] The initial global semantic map acquisition module is configured to acquire an initial global semantic map, and divide the initial global semantic map into a plurality of local subgraphs.

[0153] The grid division module is configured to perform grid division on each local subgraph according to a first preset resolution, to generate a first-level grid map corresponding to each local subgraph; the first preset resolution represents the size of each grid in the first-level grid map corresponding to each local subgraph.

[0154] The local grid subgraph generation module is configured to perform preset times of resolution reduction processing on the first-level grid map corresponding to each local subgraph based on a preset resolution level, to obtain each local grid subgraph.

[0155] Optionally, the loop detection module further comprises:

[0156] The rotation histogram determination module is configured to determine a first rotation histogram of the initial to-be-matched semantic point cloud and a second rotation histogram of the current local grid subgraph.

[0157] The rotation transformation module is configured to rotate the first rotation histogram by a preset unit rotation angle for a plurality of times, to obtain a first rotation histogram corresponding to each rotation angle.

[0158] The correlation score determination module is configured to determine a correlation score of the first rotation histogram corresponding to each rotation angle and the second rotation histogram, respectively.

[0159] The target rotation angle determination module is configured to determine a target first rotation histogram corresponding to the highest correlation score, and take a rotation angle corresponding to the target first rotation histogram as the target rotation angle.

[0160] Optionally, the to-be-matched semantic point cloud transformation module 330 comprises:

[0161] a rotation range determination unit configured to determine a rotation range of the rotation operation based on the target rotation angle, and determine a preset rotation number based on the rotation range;

[0162] a geometric center determination unit configured to obtain a first geometric center of the initial to-be-matched semantic point cloud and a second geometric center of the current local grid submap;

[0163] a translation range determination unit configured to determine a translation range of the translation operation based on the first geometric center and the second geometric center, and determine a preset translation number based on the translation range;

[0164] a to-be-matched semantic point cloud transformation unit configured to perform the rotation operation of the preset rotation number on the initial to-be-matched semantic point cloud, and perform the translation operation of the preset translation number on each rotation operation to obtain the to-be-matched semantic point cloud set.

[0165] Optionally, the matching module 340 comprises:

[0166] a first matching degree score determination module configured to perform occupancy state matching between each frame of to-be-matched semantic point cloud in the to-be-matched semantic point cloud set and the lowest level grid map corresponding to the current local grid submap, and determine a first matching degree score corresponding to each frame of to-be-matched semantic point cloud and the lowest level grid map;

[0167] a matching result generation module configured to obtain a matching result based on the first matching degree score corresponding to each frame of to-be-matched semantic point cloud and the lowest level grid map.

[0168] Optionally, the matching result generation module comprises:

[0169] a first matching degree score determination unit configured to determine a highest first matching degree score from the first matching degree score corresponding to each frame of to-be-matched semantic point cloud and the lowest level grid map;

[0170] a first matching result generation unit configured to determine a matching result as a first matching result when the highest first matching degree score is less than a preset score threshold, the first matching result indicating that there is no target to-be-matched semantic point cloud;

[0171] The second matching degree score determination unit is configured to, when the highest first matching degree score is not less than the preset score threshold, perform occupancy state matching on the highest first matching degree score corresponding to the to-be-matched semantic point cloud layer by layer starting from the lowest level grid map corresponding to the current local grid submap, to obtain a second matching degree score of the highest first matching degree score corresponding to the to-be-matched semantic point cloud and each level grid map.

[0172] The first matching result generation unit is configured to, when the second matching degree score is less than the preset score threshold, stop the occupancy state matching, and determine the matching result as the first matching result.

[0173] The target to-be-matched semantic point cloud determination unit is configured to, when the second matching degree score of the to-be-matched semantic point cloud and each level grid map is not less than the preset score threshold, determine the matching result as a second matching result, and the second matching result indicates that the highest first matching degree score corresponding to the to-be-matched semantic point cloud is taken as a target to-be-matched semantic point cloud.

[0174] Optionally, the map optimization module 350 comprises:

[0175] The pose transformation information acquisition unit is configured to acquire a rotation angle and a translation distance of the target to-be-matched semantic point cloud relative to the initial to-be-matched semantic point cloud.

[0176] The optimized pose information determination unit is configured to determine optimized pose information of the target to-be-matched semantic point cloud based on the rotation angle and the translation distance and the current local grid submap at the end of the occupancy state matching.

[0177] The map update unit is configured to update the global semantic map based on the optimized pose information of the target to-be-matched semantic point cloud.

[0178] Optionally, the loop detection device comprises:

[0179] The inverse perspective transformation unit is configured to splice the current loop-view image collected by the mobile device to obtain an inverse perspective transformation image.

[0180] The semantic segmentation unit is configured to perform semantic segmentation on the inverse perspective transformation image to obtain a to-be-matched semantic image.

[0181] The coordinate mapping unit is configured to perform coordinate mapping on the to-be-matched semantic image to obtain the initial to-be-matched semantic point cloud.

[0182] It should be noted that the apparatus provided by the above embodiments, in realizing its functions, only takes the above-mentioned division of each functional module as an example, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0183] The embodiment of the application further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to realize the steps of the above-mentioned loop detection method.

[0184] The memory can be used to store software programs and modules, and the processor can execute various functions by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access for the processor to the memory. The processor can be a central processing unit, and can also be other general-purpose processors, digital signal processors, application-specific integrated circuits, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0185] The method embodiment provided by the embodiment of the application can be executed in a computer terminal, a server or a similar computing device. Taking the running on the terminal as an example, Figure 4 is a hardware structure schematic diagram of a server running a loop detection method provided by the embodiment of the application, as Figure 4As shown, the server 400 can vary greatly in configuration and performance, and can include one or more processors (Central Processing Units, CPUs) 410 (which can include, but are not limited to, a microprocessor, a microcontroller, a programmable logic device, etc.), a memory 430 for storing data, one or more storage media 420 (such as one or more mass storage devices) for storing applications 423 and data 422. The memory 430 and the storage media 420 can be of any type known or to be developed, and can be removable or non-removable. The storage media 420 can include one or more modules, each of which can include a series of instructions for operating the server. Further, the processor 410 can be configured to communicate with the storage media 420 to execute the series of instructions in the storage media 420 on the server 400. The server 400 can also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input / output interfaces 440, and / or one or more operating systems 421, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0186] The input / output interface 440 can be configured to receive or transmit data via a network. Examples of the network can include a wireless network provided by a communication provider of the server 400. In one example, the input / output interface 440 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the input / output interface 440 can be a radio frequency (RF) module that is configured to communicate with the Internet through a wireless manner.

[0187] Those of ordinary skill in the art can understand that, Figure 4 The structure shown is merely illustrative, and does not limit the structure of the electronic device described above. For example, the server 400 can include more or less components than those shown, or have a different configuration than that shown. Figure 4 For example, the server 400 can include more or less components than those shown, or have a different configuration than that shown. Figure 4 For example, the server 400 can include more or less components than those shown, or have a different configuration than that shown.

[0188] The embodiment of the present application further provides a computer readable storage medium, wherein at least one instruction or at least one program is stored in the computer readable storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the steps of the loop detection method. In the embodiment of the present application, the computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0189] The embodiment of the present application further provides a computer readable storage medium, wherein at least one instruction or at least one program is stored in the computer readable storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the steps of the loop detection method. In the embodiment of the present application, the computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0190] The embodiment of the present application further provides a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the loop detection method provided in the various optional implementation manners.

[0191] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A loopback detection method, characterized in that, The method comprises the following steps: obtaining an initial semantic point cloud to be matched and a global semantic map; the global semantic map comprises a plurality of local grid sub-maps, and each local grid sub-map is a multi-resolution level grid map; selecting a current local grid sub-map from the plurality of local grid sub-maps; the current local grid sub-map is any local grid sub-map that has not been matched in the plurality of local grid sub-maps; performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched; the target rotation angle of the rotation operation is determined based on a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid sub-map; based on the multi-resolution level grid map corresponding to the current local grid sub-map, performing occupancy state matching on each frame of semantic point cloud to be matched in the set of semantic point clouds to be matched to obtain a matching result; in the case that the matching result indicates that there is no target semantic point cloud to be matched, performing the step of selecting a current local grid sub-map from the plurality of local grid sub-maps until the matching result indicates that there is the target semantic point cloud to be matched, and ending the occupancy state matching; updating the global semantic map based on the target semantic point cloud to be matched and the current local grid sub-map at the end of the occupancy state matching.

2. The loop detection method of claim 1, wherein, Further comprising: obtaining an initial global semantic map, and dividing the initial global semantic map into a plurality of local sub-maps; performing grid division on each local sub-map according to a first preset resolution to generate a first level grid map corresponding to each local sub-map; the first preset resolution represents the size of each grid in the first level grid map corresponding to each local sub-map; performing a preset number of resolution reduction processes on the first level grid map corresponding to each local sub-map based on a preset resolution level to obtain each local grid sub-map.

3. The loop detection method of claim 1, wherein, After selecting the current local grid sub-map from the plurality of local grid sub-maps, further comprising: determining a first rotation histogram of the initial semantic point cloud to be matched and a second rotation histogram of the current local grid sub-map; rotating the first rotation histogram by a preset unit rotation angle multiple times to obtain a first rotation histogram corresponding to each rotation angle; determining a correlation score of each rotation angle corresponding to the first rotation histogram and the second rotation histogram, respectively; determining a target first rotation histogram corresponding to the highest correlation score, and taking the rotation angle corresponding to the target first rotation histogram as the target rotation angle.

4. The loop detection method of claim 3, wherein, The method of performing a preset number of rotation operations and translation operations on the initial semantic point cloud to be matched to obtain a set of semantic point clouds to be matched comprises: determining a rotation range of the rotation operation based on the target rotation angle, and determining a preset number of rotations based on the rotation range; obtaining a first geometric center of the initial semantic point cloud to be matched and a second geometric center of the current local grid sub-map; determining a translation range of the translation operation based on the first geometric center and the second geometric center, and determining a preset number of translations based on the translation range; The initial semantic point cloud to be matched is subjected to a preset number of rotation operations, and a preset number of translation operations are performed at each rotation operation, to obtain the set of semantic point clouds to be matched.

5. The loop detection method of claim 1, wherein, The matching result includes: Each frame of the set of semantic point clouds to be matched is subjected to occupancy state matching with the lowest level grid map corresponding to the current local grid subgraph, to determine a first matching degree score of each frame of the set of semantic point clouds to be matched and the lowest level grid map; The matching result is obtained based on the first matching degree score of each frame of the set of semantic point clouds to be matched and the lowest level grid map.

6. The loop detection method of claim 5, wherein, The matching result is obtained based on the first matching degree score of each frame of the set of semantic point clouds to be matched and the lowest level grid map. The matching result is obtained based on the first matching degree score of each frame of the set of semantic point clouds to be matched and the lowest level grid map. The highest first matching degree score is determined from the first matching degree score of each frame of the set of semantic point clouds to be matched and the lowest level grid map; When the highest first matching degree score is less than a preset score threshold, the matching result is determined to be a first matching result, and the first matching result indicates that there is no target semantic point cloud to be matched; When the highest first matching degree score is not less than the preset score threshold, occupancy state matching is performed on the target semantic point cloud corresponding to the highest first matching degree score from the lowest level grid map corresponding to the current local grid subgraph layer by layer, to obtain a second matching degree score of the target semantic point cloud corresponding to the highest first matching degree score and each level grid map; wherein When the second matching degree score is less than the preset score threshold, the occupancy state matching is stopped, and the matching result is determined to be the first matching result; 7. The loop detection method of claim 1, wherein, When the second matching degree score of the target semantic point cloud and each level grid map is not less than the preset score threshold, the matching result is determined to be a second matching result, and the second matching result indicates that the target semantic point cloud corresponding to the highest first matching degree score is taken as a target semantic point cloud to be matched. The global semantic map is updated based on the target semantic point cloud to be matched and the current local network subgraph at the end of the occupancy state matching, including: The rotation angle and the translation distance of the target semantic point cloud to be matched relative to the initial semantic point cloud to be matched are obtained; The optimization pose information of the target semantic point cloud to be matched is determined based on the rotation angle and the translation distance and the current local grid subgraph at the end of the occupancy state matching; 8. The loop detection method of any of claims 1 to 7, wherein, The global semantic map is updated based on the optimization pose information of the target semantic point cloud to be matched. Further comprising: The surround view images currently collected by the mobile device are spliced to obtain inverse perspective transformation images; The inverse perspective transformation images are subjected to semantic segmentation to obtain semantic images to be matched; 9. A loopback detection apparatus, characterized by, The initial semantic point cloud to be matched is obtained by mapping the coordinates of the semantic images to be matched. Comprising: An acquisition module is configured to acquire an initial semantic point cloud to be matched and a global semantic map; The global semantic map comprises a plurality of local grid sub-maps, each of the local grid sub-maps being a grid map of a multi-resolution level; a local grid sub-map selection module configured to select a current local grid sub-map from the plurality of local grid sub-maps, the current local grid sub-map being any of the plurality of local grid sub-maps that has not been matched before; a to-be-matched semantic point cloud transformation module configured to perform a preset number of rotation operations and translation operations on the initial to-be-matched semantic point cloud to obtain a set of to-be-matched semantic point clouds, a target rotation angle of the rotation operation being determined based on a first rotation histogram of the initial to-be-matched semantic point cloud and a second rotation histogram of the current local grid sub-map; a matching module configured to perform occupancy state matching on each frame of to-be-matched semantic point cloud in the set of to-be-matched semantic point clouds based on the multi-resolution level grid map corresponding to the current local grid sub-map to obtain a matching result; in a case where the matching result indicates that there is no target to-be-matched semantic point cloud, performing the step of selecting the current local grid sub-map from the plurality of local grid sub-maps until the matching result indicates that there is the target to-be-matched semantic point cloud, and ending the occupancy state matching; a map optimization module configured to update the global semantic map based on the target to-be-matched semantic point cloud and the current local grid sub-map at the end of the occupancy state matching.

10. An electronic device, comprising: A processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the steps of the method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the steps of the method according to any one of claims 1 to 8.

Images