A centralized collaborative SLAM map construction method and system

Abstract

The present invention discloses a centralized collaborative SLAM map construction method and system, which relates to the field of map construction technology. It includes: obtaining image data of multiple areas to be measured, and converting the image data into multiple grayscale images; constructing a grayscale image processing module in the centralized collaborative SLAM, inputting multiple grayscale images into the grayscale image processing module trained by image data, obtaining the overall brightness value of each grayscale image, and if the overall brightness value of the grayscale image is less than the brightness threshold, performing image brightening processing; obtaining the clarity of each grayscale image, and if the clarity of the grayscale image is less than the clarity threshold, performing image dust removal processing; extracting the feature points and depth image data of each processed grayscale image, and fusing the feature points and depth image data of multiple processed grayscale images to obtain a map. The present invention realizes collaborative positioning and mapping in scenes with rich visual features and little illumination change, and constructs a map with high accuracy and strong real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of map construction, and in particular to a centralized collaborative SLAM map construction method and system. Background Art

[0002] Centralized collaborative SLAM (Simultaneous Localization and Mapping) includes multiple independent clients that can collect and transmit data and a central server with powerful computing power. Data is collected through the client and transmitted to the server. The server builds a server map based on the data transmitted by the client and updates it in real time to achieve real-time information sharing and collaboration.

[0003] In the prior art, the composition method of the centralized collaborative synchronous positioning and mapping system includes: selecting a suitable map model according to the actual scenario, such as a 2D grid map, a 3D point cloud map, etc. Use the SLAM algorithm to build a preliminary map based on sensor data (such as lidar, camera, etc.). Extract feature points from the sensor data, which are used for subsequent alignment and positioning. Align the sensor data at different times to the same coordinate system, and determine the position and posture of the robot. Based on the results of alignment and positioning, update the position and posture of the robot in the map, as well as the position of the feature points in the environment. Repeat until a high-quality map is built. Visualize the constructed map to better understand and analyze the environment.

[0004] The defects of the above-mentioned prior art are: the main application environment is the indoor space with rich visual features and little lighting changes, and the ability to construct scene maps for dark scenes and scenes with few environmental features is poor. Summary of the invention

[0005] Based on this, it is necessary to provide a centralized collaborative SLAM map construction method and system to address the above technical problems.

[0006] An embodiment of the present invention provides a centralized collaborative SLAM map construction method, comprising:

[0007] Acquire image data of multiple areas to be tested, and convert the image data into multiple grayscale images;

[0008] Construct grayscale image processing module in centralized collaborative SLAM;

[0009] Input multiple grayscale images into the grayscale image processing module in the centralized collaborative SLAM to obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, the image is brightened to obtain a preliminary processed grayscale image.

[0010] Obtaining the clarity of each preliminarily processed grayscale image, if the clarity of the preliminarily processed grayscale image is less than the clarity threshold, performing image dust removal processing to obtain a processed grayscale image;

[0011] The feature points and depth image data of each processed grayscale image are extracted, and the feature points and depth image data of multiple processed grayscale images are fused to construct a map.

[0012] In addition, the obtaining of the overall brightness value of each grayscale image specifically includes:

[0013] Add all the brightness values ​​and take the average value to get the average brightness avg of the grayscale image after removing the mean:

[0014]

[0015] avg = sum / width × height

[0016] Where sum represents the sum of all image grayscales after removing the mean, L is the grayscale value of the corresponding coordinate (x, y), and avg represents the average brightness of the image;

[0017] Calculate the weighted mean difference mean of the grayscale image as a whole:

[0018]

[0019] mean=total / width×height

[0020] Calculate the ratio of avg to mean cast:

[0021] cast=|avg / mean|

[0022] The cast value is used as the overall brightness value of the grayscale image to determine the overall brightness of the image; if cast>1, it means that the overall brightness of the image is abnormal, and if avg>0, it means that the image is too bright; otherwise, it means that the image is too dark; if cast<=1, the overall brightness of the image is considered normal.

[0023] In addition, the image brightening process specifically includes:

[0024] Decompose the grayscale image into RGB three-channel component images, construct three Gaussian surround functions, and convolve the component grayscale images of each channel with the Gaussian surround function to obtain a three-channel illumination estimation image;

[0025] In the logarithmic domain, the illumination estimation image is subtracted from the original image to obtain a three-channel reflection component image, and the three-channel reflection component images are merged into one image to obtain the MSR enhanced image;

[0026] Calculate the color restoration function CRF of each channel, the formula is:

[0027] C i (x,y)=β(log(αI i (x,y))-log(Σ i∈{r,g,b} I i (x,y)))

[0028] Among them, C i represents the color restoration factor of the i-th channel, which is used to adjust the ratio of the colors of the three channels. β is the gain constant, α is the controlled nonlinear intensity, and I i(x,y) Represents the image of the i-th channel;

[0029] The color restoration function CRF is multiplied by the function of the MSR enhanced image to obtain the final image enhancement components of the three channels, and the three final image enhancement components are combined to obtain the grayscale image after the MSRCR brightness enhancement, that is, the preliminary processed grayscale image.

[0030] In addition, the obtaining of the clarity of each preliminarily processed grayscale image specifically includes:

[0031] Calculate the dark channel of the preliminary processed grayscale image, the formula is:

[0032]

[0033] Among them, J C is a haze-free image, k represents the coordinates of a point in the image, traverse the r, g, and b channels, and find a minimum value in the local window Ω(x) centered on k in each channel;

[0034] Calculate the transmittance of the grayscale image, the formula is:

[0035]

[0036] in, Indicates the transmission rate, I c represents the original image, A c represents the estimated value of atmospheric light, ω is the proportionality coefficient;

[0037] The clarity is obtained through the transmittance of the grayscale image. If the transmittance of the grayscale image is lower than the set threshold value 0.25, it is determined that the clarity of the grayscale image is low.

[0038] In addition, the image dust removal process specifically includes:

[0039] Calculate a dark channel image by inputting each preliminary processed grayscale image, and determine the minimum value of each pixel in the dark channel image;

[0040] The transmission rate of each pixel is obtained by calculating the brightness ratio between the minimum value of each pixel and the maximum value in the dark channel image;

[0041] The maximum value in the dark channel image is the estimated atmospheric light. The input preliminary processed grayscale image is dust-removed and reconstructed using the estimated atmospheric light and transmission rate to obtain a processed grayscale image.

[0042] In addition, the dust removal and reconstruction of the input preliminary processed grayscale image by estimating the atmospheric light and transmission rate specifically includes:

[0043] Calculate the dark channel of the input grayscale image

[0044] Calculate the transmittance of the input grayscale image

[0045] The ambient light A is estimated by the transmittance of the input grayscale image, specifically including: arranging the pixels in the dark channel image from large to small according to brightness, and selecting the top 0.1% of the pixels; finding the highest brightness value of the pixels at the corresponding position in the original foggy image as A, or finding the average brightness of all pixels as A;

[0046] Construct the processed grayscale image:

[0047]

[0048] Where I(x) represents the foggy image, A represents the average pixel brightness, and t0 represents the lower limit of transmittance. x Represents the grayscale transmittance.

[0049] In addition, extracting feature points and depth image data of each processed grayscale image through the visual odometer submodule includes:

[0050] Traverse each processed grayscale image and select each pixel block at a time. Let the selected pixel block be P and its brightness be I. P ;

[0051] Taking pixel block P as the center, select a circular region with a radius of 3, and select 16 pixel blocks on the circle in a stacked manner;

[0052] According to experience, a threshold t is set. If there are n (n is generally set to be greater than 12) consecutive pixels in the 16 pixel blocks in the second step whose values ​​are greater than I P +t, or less than I P -t, then P can be considered as a feature point;

[0053] The depth image is directly obtained by the RGB-D camera.

[0054] In addition, the fusion of the feature points of the plurality of processed grayscale images and the depth image data specifically includes:

[0055] A local database is established based on the feature points of the multiple processed grayscale images and the depth image data, and the data in the local database is divided into two groups of image sets, one from the same source and one from different sources, according to the source;

[0056] The data from image sets of different sources are grouped in pairs, and bag-of-words data matching is performed on each group of data. The score is used to determine whether the two can be fused. If the score is greater than the threshold, the two maps are fused into the same map and divided into the same source image set.

[0057] Loop closure detection is performed on images in the image set from the same source, and when loop closures are detected, loop fusion is performed to obtain a map.

[0058] In addition, a centralized collaborative SLAM map building system includes multiple clients and servers;

[0059] The client includes: a data acquisition module, a grayscale image processing module, and an information transmission module;

[0060] The data acquisition module is used to acquire image data of multiple areas to be tested and convert the image data into multiple grayscale images;

[0061] The grayscale image processing module includes: an image brightening submodule, an image dust removal submodule, and a visual odometer submodule;

[0062] The image brightening submodule is used to obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, the image is brightened to obtain a preliminary processed grayscale image.

[0063] The image dust removal submodule is used to obtain the clarity of each preliminarily processed grayscale image. If the clarity of the preliminarily processed grayscale image is less than the clarity threshold, the image dust removal process is performed to obtain a processed grayscale image.

[0064] The visual odometer submodule is used to extract feature points and depth image data of each processed grayscale image;

[0065] The server is used to fuse the feature points of multiple processed grayscale images and the depth image data to construct a map.

[0066] In addition, the server includes: a communication module, an LCM message module, and a key frame removal module;

[0067] The communication module is used to establish a local database based on the feature points and depth image data of multiple processed grayscale images, and divide the data in the local database into two groups of image sets of the same source and different sources according to the source; the data in the image sets of different sources are grouped into two groups, and the bag-of-words data matching is performed on each group of data, and it is judged by the score whether the two can be fused. If the score passes the threshold, the two maps are fused into the same map and divided into the image set of the same source; the images in the image set of the same source are loop-fused to obtain a map;

[0068] The LCM message module is used to receive the constructed dense point cloud information and construct a dense point cloud map at the same time, and to reduce noise and compress the point cloud data in real time through a filtering algorithm. When a local loop or map fusion occurs, the dense point cloud update thread is activated to update each frame of the point cloud in real time;

[0069] The key frame removal module is used to quantify each map point and sequentially remove the data in all server databases.

[0070] Compared with the prior art, the centralized collaborative SLAM map construction method and system provided by the embodiment of the present invention have the following beneficial effects:

[0071] Acquire image data of multiple areas to be tested, and convert the image data into multiple grayscale images; construct a grayscale image processing module in the centralized collaborative SLAM, input multiple grayscale images into the grayscale image processing module trained by image data, obtain the overall brightness value of each grayscale image, if the overall brightness value of the grayscale image is less than the brightness threshold, perform image brightening processing; obtain the clarity of each grayscale image, if the clarity of the grayscale image is less than the clarity threshold, perform image dust removal processing; extract the feature points and depth image data of each processed grayscale image, and fuse the feature points and depth image data of multiple processed grayscale images to obtain a map.

[0072] The above technical solution determines whether to perform image brightening processing by calculating the overall brightness value of each grayscale image, and determines whether to perform image dust removal processing by calculating the clarity of each grayscale image. The image obtained from the test area is processed twice to make the image features more obvious, thereby realizing collaborative positioning and mapping in scenes with rich visual features and little lighting changes, solving the problem that the collaborative SLAM system is difficult to obtain feature points under harsh environmental conditions, and overcoming the defect of poor map construction ability in dark scenes and scenes with sparse environmental features, thereby constructing a map with high accuracy and strong real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 An overall architecture diagram of a centralized collaborative SLAM map building system provided in one embodiment;

[0074] Figure 2 A diagram of specific steps for image brightening in a centralized collaborative SLAM map building system provided in one embodiment;

[0075] Figure 3 A diagram of specific steps for image dust removal in a centralized collaborative SLAM map building system provided in one embodiment;

[0076] Figure 4 A fusion operation diagram of a centralized collaborative SLAM map building system provided in one embodiment;

[0077] Figure 5 A key frame data screening strategy diagram of a centralized collaborative SLAM map construction method provided in one embodiment. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0079] In one embodiment, a centralized collaborative SLAM map construction method is provided, the method comprising:

[0080] Step 1: Obtain image data of multiple areas to be tested, and convert the image data into multiple grayscale images.

[0081] Step 2: Build a grayscale image processing module in centralized collaborative SLAM.

[0082] Step 3: Input multiple grayscale images into the grayscale image processing module in the centralized collaborative SLAM.

[0083] (1) Obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, perform image brightening processing to obtain a preliminary processed grayscale image.

[0084] A. Get the overall brightness value of each grayscale image

[0085] Add all the brightness values ​​and take the average value to get the average brightness avg of the grayscale image after removing the mean:

[0086]

[0087] avg = sum / width × height

[0088] Among them, sum represents the sum of all image grayscale after removing the mean, L is the grayscale value of the corresponding coordinate (x, y), and avg represents the average brightness of the image.

[0089] Calculate the weighted mean difference mean of the grayscale image as a whole:

[0090]

[0091] mean=total / width×height

[0092] Calculate the ratio of avg to mean cast:

[0093] cast=|avg / mean|

[0094] The cast value is used as the overall brightness value of the grayscale image to determine the overall brightness of the image. If cast>1, it means that the overall brightness of the image is abnormal, and if avg>0, it means that the image is too bright; otherwise, it means that the image is too dark. If cast<=1, the overall brightness of the image is considered normal.

[0095] B. Image brightening

[0096] The grayscale image is decomposed into RGB three-channel component images, three Gaussian surround functions are constructed, and the component grayscale image of each channel is convolved with the Gaussian surround function to obtain a three-channel illumination estimation image.

[0097] In the logarithmic domain, the illumination estimation image is subtracted from the original image to obtain a three-channel reflection component image, and the three-channel reflection component images are merged into one image to obtain the MSR enhanced image.

[0098] Calculate the color restoration function CRF of each channel, the formula is:

[0099] C i (x,y)=β(log(αI i (x,y))-log(Σ i∈{r,g,b} I i (x,y)))

[0100] Among them, C i represents the color restoration factor of the i-th channel, which is used to adjust the ratio of the colors of the three channels. β is the gain constant, α is the controlled nonlinear intensity, and I i(x,y) Represents the image of the i-th channel.

[0101] The color restoration function CRF is multiplied by the function of the MSR enhanced image to obtain the final image enhancement components of the three channels, and the three final image enhancement components are combined to obtain the grayscale image after the MSRCR brightness enhancement, that is, the preliminary processed grayscale image.

[0102] (2) Calculate the clarity of each preliminarily processed grayscale image. If the clarity of the preliminarily processed grayscale image is less than the clarity threshold, perform image dust removal processing.

[0103] A. Calculate the clarity of each preliminary processed grayscale image

[0104] Calculate the dark channel of each preliminary processed grayscale image using the formula:

[0105]

[0106] Among them, J C It is a haze-free image, k represents the coordinates of a point in the image, traverses the r, g, and b channels, and finds a minimum value in the local window Ω(x) centered on k in each channel.

[0107] Calculate the transmittance of the grayscale image, the formula is:

[0108]

[0109] in, Indicates the transmission rate, I c represents the original image, A c represents the estimated value of atmospheric light, and ω is the proportionality coefficient.

[0110] The clarity is obtained through the transmittance of the grayscale image. If the transmittance of the grayscale image is lower than the set threshold value 0.25, it is determined that the clarity of the grayscale image is low.

[0111] B. Image dust removal

[0112] The dark channel image is calculated by each preliminary processed grayscale image input, and the minimum value of each pixel in the dark channel image is determined.

[0113] The transmission rate of each pixel is obtained by calculating the brightness ratio between the minimum value of each pixel and the maximum value in the dark channel image.

[0114] The maximum value in the dark channel image is the estimated atmospheric light. The input preliminary processed grayscale image is dust-removed and reconstructed using the estimated atmospheric light and transmission rate to obtain a processed grayscale image.

[0115] C. Dust removal and reconstruction

[0116] Calculate the dark channel of the input grayscale image

[0117] Calculate the transmittance of the input grayscale image

[0118] The ambient light A is estimated by the transmittance of the input grayscale image, specifically including: arranging the pixels in the dark channel image from large to small according to brightness, and selecting the top 0.1% of the pixels; searching for the highest brightness value of the pixels at the corresponding position in the original foggy images I and II as A, or finding the average brightness of all pixels as A.

[0119] Construct the processed grayscale image:

[0120]

[0121] Where I(x) represents the foggy image, A represents the average pixel brightness, and t0 represents the lower limit of transmittance. x Represents the grayscale transmittance.

[0122] (3) Extracting feature points and depth image data of each processed grayscale image, including:

[0123] Traverse each processed grayscale image and select each pixel block at a time. Let the selected pixel block be P and its brightness be I. P .

[0124] With pixel block P as the center, a circular region with a radius of 3 is selected, and 16 pixel blocks are selected on the circle in a stacked manner.

[0125] According to experience, a threshold t is set. If there are n (n is generally set to be greater than 12) consecutive pixels in the 16 pixel blocks in the second step whose values ​​are greater than I P +t, or less than I P -t, then P can be considered as a feature point. The depth image is directly obtained through RGB-D.

[0126] (4) The feature points of multiple processed grayscale images and the depth image data are fused to construct a map.

[0127] A local database is established based on the feature points of the multiple processed grayscale images and the depth image data, and the data in the local database is divided into two groups of image sets, one from the same source and one from different sources, according to the source;

[0128] The data from image sets of different sources are grouped in pairs, and bag-of-words data matching is performed on each group of data. The score is used to determine whether the two can be fused. If the score is greater than the threshold, the two maps are fused into the same map and divided into the same source image set.

[0129] Loop closure detection is performed on images in the image set from the same source, and when loop closures are detected, loop fusion is performed to obtain a map.

[0130] In one embodiment, a centralized collaborative SLAM map construction system is provided, such as Figure 1 As shown, the system includes multiple clients and servers.

[0131] The client includes: data acquisition module, grayscale image processing module, information transmission module:

[0132] 1. The data acquisition module is used to acquire image data of multiple areas to be tested and convert the image data into multiple grayscale images.

[0133] 2. The grayscale image processing module includes: image brightening submodule, image dust removal submodule, and visual odometry submodule.

[0134] (1) The image brightening submodule is used to obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, the image is brightened to obtain a preliminary processed grayscale image.

[0135] (2) The image dust removal submodule is used to obtain the clarity of each preliminarily processed grayscale image. If the clarity of the grayscale image is less than the clarity threshold, the image dust removal process is performed to obtain a processed grayscale image.

[0136] (3) The visual odometry submodule is used to extract feature points and depth image data from each processed grayscale image.

[0137] The server is used to fuse the feature points of multiple processed grayscale images and the depth image data to construct a map. The server includes: communication module, LCM message module, and key frame removal module.

[0138] The communication module is used to establish a local database based on the feature points and depth image data of multiple processed grayscale images, and divide the data in the local database into two groups of images from the same source and different sources according to their sources; the data in the image sets from different sources are grouped in pairs, and bag-of-words data matching is performed on each group of data. The score is used to determine whether the two can be fused. If the score passes the threshold, the two maps are fused into the same map and divided into the image set from the same source; the images in the image set from the same source are loop-fused to obtain a map.

[0139] The LCM message module is used to receive the constructed dense point cloud information and build a dense point cloud map at the same time, and to reduce noise and compress the point cloud data in real time through a filtering algorithm. When a local loop or map fusion occurs, the dense point cloud update thread is activated to update each frame of the point cloud in real time.

[0140] The keyframe removal module is used to quantify each map point and sequentially remove the data in all server databases.

[0141] Each client module is equipped with an RGB-D camera for real-time data acquisition and image brightening and dust removal enhancement. The central server module is mainly used for database construction, loop detection, map fusion, dense map construction, and overall optimization. The communication module mainly includes the coordinated parallel of the ROS-based message mechanism and the LCM message module. Different data are transmitted using different transmission methods through TCP and UDP to improve the transmission rate, and the server data processing speed is improved through multi-threaded processing. The overall processing flow is as follows: For each client, the visual odometer submodule with image enhancement processing is run to extract feature points and build local maps. Data with high accuracy requirements but low real-time requirements, such as travel routes, are transmitted through the ROS communication module. Feature data and point cloud data with high real-time requirements are transmitted through the LCM communication module. The server receives and processes data by subscribing to the corresponding topic, that is, it packages the packaged information sent by each client, generates key frames and updates the database, and constructs the server map and dense point cloud map at the same time. The loop detection module and map fusion module are used to perform closed-loop operations on the same sub-map and different sub-maps respectively, and the loop closure and fusion correction of the dense point cloud are performed in real time. For the processing of dense point clouds, multi-threaded operations are performed through the multi-threaded library OPENMP to ensure the real-time performance of the system, and the effect of the dense point cloud map is continuously optimized through SIM3 optimization.

[0142] Since the performance of the mobile processor carried by each client is not high, the local map maintained by each client is limited and dynamically updated to enable it to work continuously.

[0143] Aiming at the problem of image feature extraction caused by poor lighting conditions in coal mines, the present invention processes the images acquired by the sensor through MSRCR and CLAHE algorithms. Figure 2 As shown, the main steps are as follows: 1. Use the Gaussian function to low-pass filter the original image. 2. Convert the original image and the filtered image to the logarithmic domain and make a difference (i.e., subtract the low-frequency components in the image) to obtain the reflection image in the logarithmic domain. 3. Repeat steps 1 and 2 at multiple scales, and sum the reflection image in the logarithmic domain at the pixel level (i.e., the MSR step) to obtain the MSR result. 4. At the channel level, sum the original image, use it as the normalization factor of each channel, normalize the weight matrix, and convert it to the logarithmic domain to obtain the image color gain. The MSR result is recombined (multiplied) according to the weight matrix and the color gain. 5. Image restoration: The color restored image is multiplied by the gain of the image pixel value change range, plus the offset of the image pixel value change range, and the final result is obtained by automatic color level balancing. 6. The processed image is processed secondary using the CLAHE algorithm to improve the image contrast.

[0144] In the presence of dust, the image may not meet the SLAM requirements. The present invention improves the dark channel prior defogging algorithm for image processing in dusty environments. Figure 3 As shown in the figure, the main steps are as follows: 1. Preprocessing: Calculate the dark channel image based on the input image to obtain the minimum value of each pixel of the image, which is the pixel value of the dark channel image. 2. Estimation of atmospheric light: The brightest pixel in the dark channel image usually corresponds to the atmospheric light in the scene. Therefore, by finding the pixel with the highest brightness in the dark channel image, the atmospheric light can be estimated. 3. Estimation of transmission rate: By calculating the brightness ratio between each pixel and the estimated atmospheric light in the image, the transmission rate of each pixel can be obtained. Brighter areas have higher transmission rates because they are less attenuated by the atmosphere, while darker areas have lower transmission rates. 4. Dust removal and reconstruction: Use the estimated atmospheric light and transmission rate to remove dust and reconstruct the original image. After removing atmospheric scattering, the objects in the scene become clearer. By optimizing the client as above, it can be ensured that the client has higher tracking robustness and better quality of transmitted data than the unmodified client, and the server can better establish a global consistency map.

[0145] The server is the integrated processing module of the system and needs to perform major processing tasks, such as Figure 4As shown in the figure, it mainly includes the following parts: 1. Receive and process client data through the communication module, establish a local database, and run the loop detection and map fusion threads simultaneously to achieve the fusion of maps from different clients. The specific implementation steps are as follows: match the bag-of-words data of the data transmitted from different clients through the map fusion module, and judge whether the two can be fused by the score. If it passes the threshold, fuse the two frames of maps into the same map. Subsequently, all the data sold by these two clients are constructed on the fused map, and finally, bundle adjustment (BA) optimization is performed on the fused map. At the same time, the loop detection module will perform loop calculation on the maps from the same client and optimize the map after loop on the server in real time. 2. Receive the constructed dense point cloud information through the LCM message mechanism, construct a dense point cloud map, and denoise and compress the point cloud data in real time through the filtering algorithm. When local loop or map fusion occurs, activate the dense point cloud update thread to update each frame of point cloud in real time. 3. Construct a key frame culling module. Since a large amount of information is received by the server when multiple clients perform collaborative positioning, a threshold that can ensure the normal real-time operation of the program is set according to the computing power requirements of the server to maintain the real-time performance of the system. The main steps are as follows: first, quantify each map point, calculate how many KF key frames each map point is seen by, and assign different redundancies to it. When obs(MPi)≥5, it is considered that the position of this map point can be robustly estimated, so its redundancy is increased by 1. When 2<obs(MPi)<5, assign 0.9, 0.7, 0.4 to it respectively. But when obs(MPi)≤2, set its redundancy to 0, such as Figure 5 As shown, by calculating the redundancy of each frame of map point and sequentially culling all the data in the server database, the real-time performance of the server operation can be ensured.

[0146] The above embodiments only represent several implementation manners of the present invention, and the description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. A centralized collaborative SLAM map construction method, characterized in that: include: Acquire image data of multiple areas to be tested, and convert the image data into multiple grayscale images; Construct grayscale image processing module in centralized collaborative SLAM; Input multiple grayscale images into the grayscale image processing module in the centralized collaborative SLAM to obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, the image is brightened to obtain a preliminary processed grayscale image. The image brightening process specifically includes: Decompose the grayscale image into RGB three-channel component images, construct three Gaussian surround functions, and convolve the component grayscale images of each channel with the Gaussian surround function to obtain a three-channel illumination estimation image; In the logarithmic domain, the illumination estimation image is subtracted from the original image to obtain a three-channel reflection component image, and the three-channel reflection component images are merged into one image to obtain the MSR enhanced image; Calculate the color restoration function CRF of each channel, the formula is: C i (x,y)=β(log(αI i (x,y))-log(∑ i∈{r,g,b} I i (x,y))) Among them, C i represents the color restoration factor of the i-th channel, which is used to adjust the ratio of the colors of the three channels. β is the gain constant, α is the controlled nonlinear intensity, and I i(x,y) Represents the image of the i-th channel; The color restoration function CRF is multiplied by the function of the MSR enhanced image to obtain the final image enhancement components of the three channels, and the three final image enhancement components are combined to obtain the grayscale image after the MSRCR brightness enhancement, that is, the preliminary processed grayscale image; Obtaining the clarity of each preliminarily processed grayscale image, if the clarity of the preliminarily processed grayscale image is less than the clarity threshold, performing image dust removal processing to obtain a processed grayscale image; The image dust removal process specifically includes: Calculate a dark channel image by inputting each preliminary processed grayscale image, and determine the minimum value of each pixel in the dark channel image; The transmission rate of each pixel is obtained by calculating the brightness ratio between the minimum value of each pixel and the maximum value in the dark channel image; The maximum value in the dark channel image is the estimated atmospheric light. The input preliminary processed grayscale image is dust-removed and reconstructed by using the estimated atmospheric light and transmission rate to obtain a processed grayscale image. Extracting feature points and depth image data of each processed grayscale image, and fusing the feature points and depth image data of multiple processed grayscale images to construct a map; The fusing of the feature points of the plurality of processed grayscale images and the depth image data specifically includes: A local database is established based on the feature points of the multiple processed grayscale images and the depth image data, and the data in the local database is divided into two groups of image sets, one from the same source and one from different sources, according to the source; The data from image sets of different sources are grouped in pairs, and bag-of-words data matching is performed on each group of data. The score is used to determine whether the two can be fused. If the score is greater than the threshold, the two maps are fused into the same map and divided into the same source image set. Loop detection is performed on images in the image set from the same source, and when loops are detected, loop fusion is performed to obtain a map.

2. A centralized collaborative SLAM map construction method as claimed in claim 1, characterized in that: The obtaining of the overall brightness value of each grayscale image specifically includes: Add all the brightness values ​​and take the average value to get the average brightness avg of the grayscale image after removing the mean: avg = sum / width × height Where sum represents the sum of all image grayscales after removing the mean, L is the grayscale value of the corresponding coordinate (x, y), and avg represents the average brightness of the image; Calculate the weighted mean difference mean of the grayscale image as a whole: mean = total / width × height Calculate the ratio of avg to mean cast: cast=|avg / mean| The cast value is used as the overall brightness value of the grayscale image to determine the overall brightness of the image; if cast>1, it means that the overall brightness of the image is abnormal, and if avg>0, it means that the image is too bright; otherwise, it means that the image is too dark; if cast<=1, the overall brightness of the image is considered normal.

3. A centralized collaborative SLAM map construction method as claimed in claim 1, characterized in that: The obtaining of the clarity of each preliminarily processed grayscale image specifically includes: Calculate the dark channel of the preliminary processed grayscale image, the formula is: Among them, J C is a haze-free image, k represents the coordinates of a point in the image, traverse the r, g, and b channels, and find a minimum value in the local window Ω(x) centered on k in each channel; Calculate the transmittance of the grayscale image, the formula is: in, Indicates the transmission rate, I c represents the original image, A c represents the estimated value of atmospheric light, ω is the proportionality coefficient; The clarity is obtained through the transmittance of the grayscale image. If the transmittance of the grayscale image is lower than the set threshold value 0.25, it is determined that the clarity of the grayscale image is low.

4. A centralized collaborative SLAM map construction method as claimed in claim 1, characterized in that: The step of performing dust removal and reconstruction on the input preliminary processed grayscale image by using the estimated atmospheric light and transmission rate specifically includes: Calculate the dark channel of the input grayscale image Calculate the transmittance of the input grayscale image The ambient light A is estimated by the transmittance of the input grayscale image, specifically including: arranging the pixels in the dark channel image from large to small according to brightness, and selecting the top 0.1% of the pixels; finding the highest brightness value of the pixels at the corresponding position in the original foggy image as A, or finding the average brightness of all pixels as A; Construct the processed grayscale image: Among them, I(x) represents the foggy image, A represents the average brightness of the pixel, and t0 represents the lower limit of the transmittance. x Represents the grayscale transmittance.

5. A centralized collaborative SLAM map construction method as claimed in claim 1, characterized in that: The extracting of feature points and depth image data of each processed grayscale image specifically includes: Traverse each processed grayscale image and select each pixel block at a time. Let the selected pixel block be P and its brightness be I. P ; Taking pixel block P as the center, select a circular region with a radius of 3, and select 16 pixel blocks on the circle in a stacked manner; According to experience, a threshold t is set. If there are n (n is generally set to be greater than 12) consecutive pixels in the 16 pixel blocks in the second step whose values ​​are greater than I P +t, or less than I P -t, then P can be considered as a feature point; Depth image data is directly obtained through an RGB-D camera.

6. A centralized collaborative SLAM map construction system based on the centralized collaborative SLAM map construction method according to any one of claims 1 to 5, characterized in that: Includes multiple clients and servers; The client includes: a data acquisition module, a grayscale image processing module, and an information transmission module; The data acquisition module is used to acquire image data of multiple areas to be tested and convert the image data into multiple grayscale images; The grayscale image processing module includes: an image brightening submodule, an image dust removal submodule, and a visual odometer submodule; The image brightening submodule is used to obtain the overall brightness value of each grayscale image. If the overall brightness value of the grayscale image is less than the brightness threshold, the image is brightened to obtain a preliminary processed grayscale image. The image dust removal submodule is used to obtain the clarity of each preliminarily processed grayscale image. If the clarity of the preliminarily processed grayscale image is less than the clarity threshold, the image dust removal process is performed to obtain a processed grayscale image. The visual odometer submodule is used to extract feature points and depth image data of each processed grayscale image; The server is used to fuse the feature points of multiple processed grayscale images and the depth image data to construct a map.

7. A centralized collaborative SLAM map construction system as claimed in claim 6, characterized in that: The server includes: a communication module, an LCM message module, and a key frame removal module; The communication module is used to establish a local database based on the feature points and depth image data of multiple processed grayscale images, and divide the data in the local database into two groups of image sets of the same source and different sources according to the source; the data in the image sets of different sources are grouped into two groups, and the bag-of-words data matching is performed on each group of data, and it is judged by the score whether the two can be fused. If the score passes the threshold, the two maps are fused into the same map and divided into the image set of the same source; the images in the image set of the same source are loop-fused to obtain a map; The LCM message module is used to receive the constructed dense point cloud information and construct a dense point cloud map at the same time, and to reduce noise and compress the point cloud data in real time through a filtering algorithm. When a local loop or map fusion occurs, the dense point cloud update thread is activated to update each frame of the point cloud in real time; The key frame removal module is used to quantify each map point and sequentially remove the data in all server databases.

Images