A Mobile Robot Relocalization Method Based on Global Descriptors

Abstract

The present invention discloses a mobile robot relocalization method based on global descriptors. In the relocalization stage, through preprocessing the data of the omnidirectional camera sensor system on the mobile robot, a local dense point cloud containing semantic information is obtained; a global descriptor is created for the above local dense point cloud, and it is matched and extended with a dense point cloud sub-map set, and the sub-map with the highest similarity score among them is used as the target dense point cloud; through point cloud registration of the local dense point cloud and the target dense point cloud, the relocalization result of the mobile robot is obtained. The invention fully integrates the semantic information and metric information provided by the omnidirectional camera sensor system, and achieves accurate and robust relocalization effects for mobile robots in complex environments with large-scale ranges, across indoor and outdoor scenarios, and in scenarios with similar geometric features.

Description

Technical Field

[0001] The present invention relates to the technical fields of robotics and map processing, and particularly to a mobile robot relocalization method based on global descriptors. Background Art

[0002] With the development and progress of science and technology, the application fields of mobile robots have gradually expanded, and the working environments they face have become increasingly complex. At present, autonomous mobile robots have been deployed in challenging open environments for various types of extended operations, such as logistics robots and scenic area sightseeing robots. During the operation of the robot, autonomous positioning and navigation are important steps for the system to cope with the problems arising from various complex environments, and how to enable the mobile robot to quickly relocalize is the first step in solving this problem.

[0003] In the application scenarios of mobile robots, the existing relocalization methods have the following problems: The method based on the positioning information of the global satellite navigation system is easily blocked indoors, resulting in incorrect pose observation information output; The method based on three-dimensional point cloud registration is difficult to return to the correct pose in areas with similar geometric features and is prone to falling into local minima; The method based on visual tags requires a large number of pre-calibrated visual tags to be deployed in the environment, greatly increasing the workload of map construction; The method based on visual features has problems with the observation direction, and a large amount of computing resources are required to achieve the extraction, storage, and matching of omnidirectional visual features. Therefore, it is of great significance to propose a method that can perform efficient, stable, and accurate relocalization tasks in complex environments with large-scale ranges, across indoor and outdoor scenarios, and with similar geometric feature scenarios to improve the autonomous navigation ability of mobile robots. Summary of the Invention

[0004] In view of this, the present invention provides a mobile robot relocalization method based on global descriptors to solve the problem that the prior art cannot perform efficient, stable, and accurate relocalization tasks in complex environments with large-scale ranges, across indoor and outdoor scenarios, and with similar geometric feature scenarios.

[0005] A mobile robot relocalization method based on global descriptors includes:

[0006] Step 1: Three-dimensional map construction, segmentation, and description methods, specifically including:

[0007] Step S110: Obtain the omnidirectional image data collected by the omnidirectional camera system and the positioning information obtained by the integrated navigation system. After preprocessing, obtain the local dense point cloud map result containing semantic information and the pose transformation between the current omnidirectional image frame and the previous omnidirectional image frame.

[0008] Step S120: Perform fusion processing on the local dense point cloud map and all previously obtained local dense point cloud maps according to the pose transformation to construct a global dense point cloud map with semantic information;

[0009] Step S130: Perform rasterization segmentation on the global dense point cloud map to divide it into dense point cloud subgraphs with semantic information; and create global descriptors for these dense point cloud subgraphs respectively, specifically:

[0010] Divide the length and width of the global dense point cloud map in the horizontal direction into m and n respectively, then the global dense point cloud map will be rasterization segmented into a subgraph set M = {M i,j |i ∈ [0, m], j ∈ [0, n]};

[0011] For each dense point cloud subgraph M i,j , cluster the point clouds with the same semantic description therein, and generate a semantic description graph G using the Euclidean distance between the clustered point clouds as a criterion i,j ; Each point cloud clustering result in this description graph is a vertex of the semantic description graph G i,j , and if the Euclidean distance between the clustered point clouds is less than the set threshold, a connection is drawn between the corresponding vertices in the description graph to represent the connection relationship;

[0012] Use the random walk description method for the semantic description graph G i,j to generate the global descriptor V of the dense point cloud subgraph i,j , obtaining a p×q-dimensional matrix, where p represents the number of random walks and q represents the depth of the random walk. Each element in the matrix is the semantic attribute index in the semantic information corresponding to the vertex reached by the random walk; use the above method for the dense point cloud subgraph set M to create the corresponding global descriptor subset

[0013] Step Two: Relocalization method, specifically including the following steps:

[0014] Step S210: Obtain the panoramic camera image data collected by the panoramic camera system set in the mobile robot, perform preprocessing operations to obtain a local dense point cloud S with semantic information, and create a global descriptor V according to the method of Step S130 S ;

[0015] Step S220: Match the global descriptor V S with the global descriptors of all dense point cloud subgraphs obtained in the map construction stage to obtain the descriptor with the highest similarity to the global descriptor of the local dense point cloud S, specifically including:

[0016] Match the global descriptor V SMatch with each element in the global description subset That is, for each global description descriptor V i,j , when matching, search for the number of rows with the same elements in the two matrices, and normalize it with the total number of rows p as the similarity score between the global description descriptor V S and the global description descriptor V i,j ; Traverse each element in the global description subset to obtain the descriptor with the highest similarity to the global description descriptor of the local dense point cloud, and determine the corresponding dense point cloud subgraph, denoted as M target ;

[0017] Step S230: Perform point cloud registration on the dense point cloud subgraph M target determined in step S220 and the local dense point cloud S to obtain the pose of the mobile robot in the global dense point cloud map coordinate system and achieve relocalization.

[0018] Furthermore, after determining the corresponding dense point cloud subgraph in step 220, merge the dense point cloud subgraph with several subgraphs in its neighborhood to obtain multiple extended subgraphs; re-extract the global description descriptors for the extended subgraphs according to the method in step S130 and calculate the similarity score between the global description descriptor V S corresponding to the local dense point cloud S, where the dense point cloud subgraph with the largest score is the dense point cloud subgraph M target .

[0019] Preferably, the specific method for obtaining multiple extended subgraphs includes:

[0020] Let the dense point cloud subgraph determined in step 220 be M i,j , and the set of extended dense point cloud subgraphs includes: M i,j +M i-1,j , M i,j +M i+1,j , M i,j +M i,j-1 , M i,j +M i,j+1 , M i,j +M i-1,j +M i,j-1 +M i-1,j-1 , M i,j +M i+1,j +M i,j-1 +M i+1,j-1 , M i,j +M i-1,j +M i,j+1 +M i-1,j+1 and M i,j +M i+1,j +M i,j+1 +M i+1,j+1;

[0021] Among them, the '+' indicates the merging of adjacent sub - maps.

[0022] Preferably, in the step S110, the pre - processing includes: extracting feature points from the panoramic image data, and using the co - visible area of adjacent cameras in the panoramic camera system and the matching results of feature points in adjacent cameras to establish corresponding 3D map points for these feature points; for pixels where feature points are not extracted, pixels with similar features are found on their co - visible cameras, thereby establishing a local dense point cloud map; combining the local dense point cloud map with the semantic segmentation result of the panoramic image data to obtain a local dense point cloud map with semantic information.

[0023] Preferably, in the step S110, synchronize and align the timestamp of the positioning data output by the integrated navigation system with the timestamp of the current panoramic image data, obtain the pose transformation relationship of the integrated navigation system between the current panoramic image frame and the previous panoramic image frame, and obtain the initial pose transformation value between the current panoramic image frame and the previous panoramic image frame according to the external parameter relationship between the panoramic camera system and the integrated navigation system; jointly optimize the initial pose transformation value and the feature point matching result between the current panoramic image frame and the previous panoramic image frame to obtain the pose transformation between the current panoramic image frame and the previous panoramic image frame.

[0024] Preferably, in the step S120, the process of fusing the local dense point cloud map and all previously obtained local dense point cloud maps includes: generating a global initial dense point cloud map according to the local dense point cloud map, all previously obtained local dense point cloud maps and their corresponding pose transformation relationships, and performing down - sampling processing on the initial map to obtain a global dense point cloud map with semantic information.

[0025] The present invention has the following beneficial effects:

[0026] The present invention provides a mobile robot re - localization method based on global descriptors. This localization method constructs a global dense point cloud map with semantic information, rasterizes it into a dense point cloud sub - atlas, and uses global descriptors to describe each dense point cloud sub - map in the set to create a global descriptor subset. Through the local dense point cloud processed on the mobile robot, its corresponding global descriptor and point cloud geometric attributes, high - efficiency, stability, and accurate re - localization tasks can be carried out in complex environments with large - scale ranges, across indoor and outdoor scenes, and scenes with similar geometric features, improving the autonomous navigation ability of the mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic flow chart of a 3D map construction, segmentation, and description method based on global descriptors in the present invention.

[0028] Figure 2 This is a schematic flowchart of a method for relocating using global descriptors and dense point clouds in the present invention.

[0029] Figure 3 This is a schematic diagram of an extended sub - graph of the present invention.

[0030] Figure 4 This is a structural block diagram of a mobile robot relocating device based on global descriptors provided in the present invention.

[0031] Figure 5 This is a structural block diagram of a computer device provided in the present invention.

[0032] Figure 6 This is a structural block diagram of a computer - readable storage medium provided in the present invention. Detailed implementation manners

[0033] The following combines the accompanying drawings and gives examples to describe the present invention in detail.

[0034] The present invention provides a method for relocating a mobile robot based on global descriptors, including:

[0035] Step 1: Methods for three - dimensional map construction, segmentation, and description. Refer to Figure 1 , specifically including:

[0036] Step S110: Obtain the panoramic image data collected by the panoramic camera system and the positioning information obtained by the integrated navigation system. After pre - processing, obtain the feature - point extraction result of the current panoramic image frame, the local dense point - cloud map result containing semantic information, and the pose transformation between the current panoramic image frame and the previous panoramic image frame.

[0037] Among them, for the panoramic image data collected by the panoramic camera system, feature - point extraction and semantic segmentation are performed. Triangulation operations are carried out using the extrinsic parameter relationship between adjacent cameras in the panoramic camera system and the feature - point matching results to establish three - dimensional map points corresponding to the feature points. For pixels that do not have feature points extracted, pixels with similar features are found on their co - visible cameras using the epipolar search method, and triangulation operations are carried out again using the extrinsic parameter relationship between adjacent cameras to establish a local dense point - cloud map corresponding to the panoramic image data. Using the semantic segmentation result of the panoramic image data, the semantic attributes corresponding to the pixels are mapped into the local dense point - cloud map, that is, each point in the point - cloud map contains position information and semantic attributes, and a local dense point - cloud map containing semantic information can be obtained.

[0038] For the positioning data output by the integrated navigation system, synchronize and align its timestamp with the timestamp of the surround-view image data to obtain the pose transformation relationship between the current surround-view image frame and the previous surround-view image frame, and obtain the initial value of the pose transformation between the current surround-view image frame and the previous surround-view image frame according to the extrinsic parameter relationship between the surround-view camera system and the integrated navigation system.

[0039] Jointly optimize the initial value of the pose transformation and the feature point matching result between the current surround-view image frame and the previous surround-view image frame, and minimize the reprojection error of the corresponding map points of the matching feature points in the two frames on the current surround-view image frame to obtain a more accurate and stable pose transformation between the current surround-view image frame and the previous surround-view image frame. Optionally, this operation can further improve the accuracy of the local dense point cloud map by fusing lidar sensor data.

[0040] Step S120: Perform fusion processing on the local dense point cloud map and all previously obtained local dense point cloud maps to construct a global dense point cloud map containing semantic information.

[0041] Among them, perform fusion processing on the local dense point cloud map containing semantic information obtained at the current moment and all previously obtained local dense point cloud maps containing semantic information. According to the local dense point cloud map, all previously obtained local dense point cloud maps and their corresponding pose transformation relationships, splice the local point cloud maps into a global point cloud map. The splicing process reduces the splicing error through methods such as point cloud alignment and global optimization, and performs downsampling processing on the optimized global dense point cloud map.

[0042] Step S130: Perform rasterization segmentation on the global dense point cloud map, divide it into dense point cloud subgraphs containing semantic information, and create global descriptors for these dense point cloud subgraphs respectively.

[0043] Specifically, divide the length and width of the global dense point cloud map in the horizontal direction into m and n respectively, without considering the segmentation in the height direction. Then the global dense point cloud map will be rasterized and segmented into a subgraph set M = {M i,j |i ∈ [0, m], j ∈ [0, n]} composed of m×n dense point cloud subgraphs.

[0044] Create a global descriptor for each subgraph in the dense point cloud subgraph set. Taking the dense point cloud subgraph M i,j as an example, cluster the point clouds with the same semantic description in it, and generate a semantic description graph G i,j using the Euclidean distance between the clustered point clouds as a criterion. Each point cloud clustering result in this description graph is the semantic description graph G i,jVertices, and if the Euclidean distance between the clustered point clouds is less than the set threshold, a connection is represented by a line between the corresponding vertices in the description graph; no connection is set between vertices greater than or equal to the set threshold.

[0045] For the semantic description graph G i,j Use the random walk description method to generate the global descriptor V of the dense point cloud subgraph i,j , this descriptor is a p×q-dimensional matrix, where p represents the number of random walks, q represents the depth of the random walk, and each element in the matrix is the semantic attribute index in the semantic information corresponding to the vertex reached by the random walk. Use the above method for the dense point cloud subgraph set M to create the corresponding global descriptor subset

[0046] Refer to Figure 2 , Figure 2 shows a schematic flow diagram of a relocalization method using the global descriptor and the dense point cloud. The following will be elaborated in detail for Figure 2 the shown process. The relocalization method specifically includes the following steps:

[0047] Step S210: Obtain the panoramic camera image data collected by the panoramic camera system set in the mobile robot, perform preprocessing operations to obtain the local dense point cloud S with semantic information, and create the global descriptor V according to the method of step S130 S .

[0048] Among them, the pixels in the panoramic camera image data are respectively found for pixels with similar features on their co-view cameras using the epipolar search method, and triangulation operations are performed using the extrinsic parameter relationship of adjacent cameras to establish the local dense point cloud corresponding to the panoramic image data. Using the semantic segmentation result of the panoramic image data, map the semantic attributes corresponding to the pixels to the local dense point cloud, and then the local dense point cloud S with semantic information can be obtained. Optionally, this operation can further improve the accuracy of the local dense point cloud map by fusing lidar sensor data.

[0049] Cluster the point clouds with the same semantic description in the local dense point cloud S, and use the Euclidean distance between the clustered point clouds as a criterion to generate a semantic description graph. Use the random walk description method for the semantic description graph to generate the global descriptor V of the local dense point cloud S S .

[0050] Step S220: Match the global descriptor V S with the global descriptors of all dense point cloud subgraphs obtained in the map construction stage to obtain the descriptor with the highest similarity to the global descriptor of the local dense point cloud S, specifically including:

[0051] Match the global descriptor V SMatch with each element in the global description subset For example, take the global descriptor V in the global description subset i,j As an example, during matching, search for the number of rows with the same elements in the two matrices, and normalize it by the total number of rows p as the similarity score between the global descriptor V S And the global descriptor V i,j Traverse each element in the global description subset To obtain the descriptor with the highest similarity to the global descriptor of the local dense point cloud.

[0052] Step S230: Expand the dense subgraph corresponding to the above descriptor with the highest similarity, extract the descriptor again, and calculate the similarity score. The dense point cloud subgraph corresponding to the descriptor with the highest similarity is the target dense point cloud subgraph, specifically:

[0053] Let the dense point cloud subgraph corresponding to the descriptor with the highest similarity be M i,j And expand it. The expanded set of dense point cloud subgraphs should include:

[0054]

[0055] Where + represents the merging of adjacent subgraphs. The dense point cloud subgraphs included in the expanded set of dense point cloud subgraphs are as shown in Figure 3 Shown.

[0056] Re-extract the global descriptor for each subgraph in the set of dense point cloud subgraphs and calculate the similarity score between it and the global descriptor V S Corresponding to the local dense point cloud S. The dense point cloud subgraph with the largest score is the target dense point cloud subgraph M target .

[0057] Step S240: Perform point cloud registration on the target dense point cloud subgraph and the local dense point cloud to obtain the pose of the mobile robot in the global dense point cloud map coordinate system and achieve relocalization, specifically:

[0058] Use the centroid of the target dense point cloud subgraph M target As the initial displacement value, set the initial pitch angle and initial roll angle to 0°, and set the initial yaw angle to N equally divided values respectively. Use these N poses as the initial poses for registering the local dense point cloud S. Apply these N initial poses to the process of registering the local dense point cloud S to the target dense point cloud subgraph M target Using the normal distribution transformation, and select the pose with the smallest registration error as the pose of the mobile robot in the global dense point cloud map coordinate system.

[0059] As shown in Figure 4As shown in the figure, the present invention also provides a mobile robot relocalization device, including: a sensor data acquisition module 310, a data processing module 320, a map construction, segmentation, and description module 330, and a relocalization module 340. Among them, the sensor acquisition module 310 is used to acquire omnidirectional image data and integrated navigation system positioning data. The sensor is rigidly connected to the acquisition device, and data transmission is realized through a communication connection to the system, and it is time-synchronized by the same clock source; the data processing module 320 is used to process the sensor data acquired by the sensor data acquisition module, including timestamp synchronization alignment, image semantic understanding, image feature point extraction, matching and triangulation, dense point cloud three-dimensional reconstruction, multi-sensor data fusion, etc.; the map construction, segmentation, and description module 330 is used to splice, fuse, and downsample the local dense point cloud subgraph containing semantic information to generate a global dense point cloud map, and generate a dense point cloud subgraph set and a global description subset for the global dense point cloud map through rasterization segmentation and a description method based on global descriptors. The relocalization module 340 is used to perform image semantic understanding and dense point cloud three-dimensional reconstruction on the omnidirectional camera data acquired on the mobile robot to generate a local dense point cloud containing point cloud information and describe it with global descriptors, match the global descriptors corresponding to the local dense point cloud with the global description subset generated by module 330 and calculate the similarity score. Expand the dense point cloud subgraph corresponding to the descriptor with the largest similarity score in the global description subset and recalculate the similarity score to obtain the target dense point cloud subgraph for point cloud registration. Perform point cloud registration on the local dense point cloud and the target dense point cloud subgraph to achieve the relocalization of the mobile robot.

[0060] Referring to Figure 5 , which shows a structural block diagram of a computer device 400 provided by an embodiment of the present application. The computer device 400 in the present application may include one or more of the following components: a processor 410, a memory 420, and one or more application programs, where one or more application programs are stored in the memory 420 and are configured to be executed by one or more processors 410, and one or more programs are configured to execute the methods described in the foregoing method embodiments.

[0061] The processor 410 may include one or more processing cores, which are connected to various parts within the entire computer device through various interfaces and lines. By running or executing programs stored in the memory 420 and calling data stored in the memory 420, it performs various functions of the computer device and processes data. Optionally, the processor 410 may be implemented in at least one hardware form of a Programmable Logic Array (PLA), a Digital Signal Processing (DSP), or a Field-Programmable Gate Array (FPGA). The processor 410 may integrate a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU). Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering visual content and participating in intensive calculations.

[0062] The memory 420 may include a Random Access Memory (RAM) and may also include a Read-Only Memory (ROM). The memory 420 can be used to store programs, code, or code sets. The memory 420 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing the above-described method embodiments, etc. The data storage area may also store data created during the use of the computer device (such as temporary variables, pre-allocated variable spaces), etc.

[0063] Refer to Figure 6 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable storage medium 500, and the program code can be called by the processor to execute the methods described in the above method embodiments. The computer-readable storage medium 500 may be an electronic memory such as a flash memory, an Electrically Erasable Programmable Read-Only Memory (EEPROM), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 500 includes a non-volatile computer-readable medium. The computer-readable storage medium 500 has a storage space for the program code 510 for executing any method steps in the above methods. These program codes can be read out from or written into one or more computer program products.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A mobile robot relocalization method based on global descriptors, characterized in that, it includes: Step 1. Three-dimensional map construction, segmentation, and description method, specifically including: Step S110: Obtain the panoramic image data collected by the panoramic camera system and the positioning information obtained by the integrated navigation system. After preprocessing, obtain the local dense point cloud map result with semantic information and the pose transformation between the current panoramic image frame and the previous panoramic image frame; Step S120: Perform fusion processing on the local dense point cloud map and all previously obtained local dense point cloud maps according to the pose transformation to construct a global dense point cloud map with semantic information; Step S130: Perform rasterization segmentation on the global dense point cloud map and divide it into dense point cloud subgraphs with semantic information; and create global descriptors for these dense point cloud subgraphs respectively, specifically: The length and width of the global dense point cloud map in the horizontal direction are divided into m and n respectively, then the global dense point cloud map will be rasterized and divided into a set of subgraphs M = {M i,j | i ∈ [0, m], j ∈ [0, n]}; For each dense point cloud sub - graph M i,j , cluster the point clouds with the same semantic description among them, and generate a semantic description graph G using the Euclidean distance between the clustered point clouds as a criterion i,j ; the clustering result of each point cloud in this description graph is the vertex of the semantic description graph G i,j ; if the Euclidean distance between the clustered point clouds is less than the set threshold, connect the corresponding vertices in the description graph to represent the connection relationship For the semantic description graph G i,j Use the random walk description method to generate the global descriptor V of the dense point cloud subgraph i,j , obtaining a p×q-dimensional matrix, where p represents the number of random walks and q represents the depth of the random walk. Each element in the matrix is the semantic attribute index in the semantic information corresponding to the vertex reached by the random walk; use the above method for the dense point cloud subgraph set M to create the corresponding global description subset Step 2. Relocalization method, specifically including the following steps: Step S210: Obtain the omnidirectional camera image data collected by the omnidirectional camera system set in the mobile robot, perform preprocessing operations to obtain a local dense point cloud S containing semantic information, and create a global descriptor V according to the method in step S130 S ; Step S220: Compare the global descriptor V S with the global descriptors of all dense point cloud subgraphs obtained in the map construction phase to obtain the descriptor with the highest similarity to the global descriptor of the local dense point cloud S. Specifically, it includes: Match the global descriptor V S with each element in the global descriptor subset . That is, for each global descriptor V i,j , when matching, search for the number of rows with the same elements in the two matrices, and normalize it by the total number of rows p as the similarity score between the global descriptor V S and the global descriptor V i,j . Traverse each element in the global descriptor subset to obtain the descriptor with the highest similarity to the global descriptor of the local dense point cloud, and determine the corresponding dense point cloud subgraph, denoted as M target ; Step S230: Perform point cloud registration on the dense point cloud sub - map M determined in step S220 target and the local dense point cloud S to obtain the pose of the mobile robot in the global dense point cloud map coordinate system, thereby achieving relocalization.

2. A mobile robot relocalization method based on global descriptors according to claim 1, characterized in that, After determining its corresponding dense point cloud sub - graph in step 220, merge the dense point cloud sub - graph with several sub - graphs in its neighborhood to obtain multiple extended sub - graphs; re - extract the global descriptors of the extended sub - graphs according to the method of step S130 and calculate the similarity score between the global descriptors V corresponding to the local dense point cloud S. Among them, the dense point cloud sub - graph with the largest score is the dense point cloud sub - graph M S target .​ 3. A mobile robot relocalization method based on global descriptors according to claim 2, characterized in that, The specific method for obtaining multiple extended subgraphs includes: Let step 220 determine that its corresponding dense point cloud sub - graph is M i,j The expanded set of dense point cloud sub - graphs includes: M i,j +M i-1,j ,M i,j +M i+1,j ,M i,j +M i,j-1 ,M i,j +M i,j+1 ,M i,j +M i-1,j +M i,j-1 +M i-1,j-1 ,M i,j +M i+1,j +M i,j-1 +M i+1,j-1 ,M i,j +M i-1,j +M i,j+1 +M i-1,j+1 and M i,j +M i+1,j +M i,j+1 +M i+1,j+1 ; where + represents the merging of adjacent subgraphs.

4. A mobile robot relocalization method based on global descriptors according to claim 1, characterized in that, In step S110, the preprocessing includes: extracting feature points from the panoramic image data, and using the common view area of adjacent cameras in the panoramic camera system and the matching results of feature points in adjacent cameras to establish corresponding three-dimensional map points for these feature points; for pixels where feature points are not extracted, find pixels with similar features on their common view cameras, thereby establishing a local dense point cloud map; combining the local dense point cloud map with the semantic segmentation result of the panoramic image data to obtain a local dense point cloud map with semantic information.

5. A mobile robot relocalization method based on global descriptors according to claim 1, characterized in that, In step S110, synchronize and align the timestamp of the positioning data output by the integrated navigation system with the timestamp of the current panoramic image data to obtain the pose transformation relationship of the integrated navigation system between the current panoramic image frame and the previous panoramic image frame, and obtain the initial pose transformation value between the current panoramic image frame and the previous panoramic image frame according to the external parameter relationship between the panoramic camera system and the integrated navigation system; Jointly optimize the initial pose transformation value and the feature point matching result between the current panoramic image frame and the previous panoramic image frame to obtain the pose transformation between the current panoramic image frame and the previous panoramic image frame.

6. A mobile robot relocalization method based on global descriptors according to claim 1, characterized in that, In the step S120, the process of fusing the local dense point cloud map and all previously obtained local dense point cloud maps includes: generating a global initial dense point cloud map according to the local dense point cloud map, all previously obtained local dense point cloud maps, and their corresponding pose transformation relationships, and performing downsampling processing on the initial map to obtain a global dense point cloud map with semantic information.

Images