Slam map splicing method and device based on deep network, equipment and medium

Abstract

The invention relates to a slab map splicing method and device based on a deep network, equipment and a medium. The method comprises the following steps: acquiring a pair of slab map images to be spliced; performing matrix processing on the slam map image, and converting the slam map image into a single-channel binary image matrix; obtaining a user instruction to determine whether to select a matching area, if so, taking the single-channel binary image matrix of the corresponding part of the matching area selected by the user as the input of the deep network, otherwise, directly taking the complete single-channel binary image matrix as the input of the deep network; performing matrix feature extraction by using a deep network, and outputting a homography transformation matrix between the two rectangular input images; and according to the homography transformation matrix, determining the position of each pixel point in the first single-channel binary image matrix in the second single-channel binary image matrix, and completing image stitching. Compared with the prior art, the method has the advantages of reducing the cost and difficulty of large-scene mapping and the like.

Description

Technical Field

[0001] The present invention relates to the field of SLAM mapping, and in particular to a SLAM map splicing method, device, equipment and medium based on a deep network. Background Art

[0002] With the development of science and technology, robotics has been widely used in various fields, among which map construction and navigation are important components of robotics. Map construction refers to the robot obtaining environmental information through sensors and constructing a map of the environment based on this information to facilitate path planning and navigation. However, due to the complexity of the environment and the limitations of the algorithm, there are often errors and omissions when the robot constructs large-scene maps, which brings challenges to the robot's navigation and operation. At the same time, for the situation where local mapping fails or the mapping effect of a certain area is not good, global reconstruction is very time-consuming and labor-intensive, so the map stitching algorithm is very important. It only needs to rebuild the map in the area where the map effect is not good, and then use the map stitching algorithm to stitch the two maps into a large map, which greatly reduces the workload and reduces the algorithm pressure, while constructing a large scene map that meets the accuracy requirements.

[0003] With the development of deep learning technology, convolutional neural networks have been widely used in the field of computer vision. Traditional image processing technology can only focus on local features of images and is not robust enough to noise, illumination changes, scale changes, and rotations. Performance may be poor in complex scenes. In addition, it requires manual design of extraction algorithms and does not have good generalization. Deep learning models can capture the global information of images, process large-scale and high-dimensional data, and perform well in processing complex and ever-changing image tasks.

[0004] Existing map processing technology mainly extracts feature points from the map and then matches these feature points to achieve map stitching. However, this method will result in sparse feature points when the map is relatively empty, and the maps saved by the robot after two mapping operations will inevitably be too different, which may cause matching errors and affect the accuracy of map stitching. The deep model network technology mainly trains the neural network so that the network can automatically learn the correspondence between images, thereby achieving image registration and stitching. This method does not require manual extraction of feature points, so accurate registration results can be obtained even when the feature points are sparse and the map differences are relatively large.

[0005] CN111709880B discloses a multi-channel image stitching method based on an end-to-end neural network, and the system includes the following steps: S1: pre-estimating the adjacency relationship between multiple images through an adjacent adjacency graph estimator, calculating the homography matrix between two images through an adaptive convolutional neural network CNN, and determining whether the two are adjacent, and determining the adjacency weight between the two, and constructing the adjacency graph between multiple images by determining the adjacency weight between two images; S2: determining the optimal base plane image according to the adjacency graph between multiple images, and selecting all images adjacent to the optimal base plane image based on the optimal base plane image; S3: projecting all images adjacent to the optimal base plane image to the optimal base plane through multiple homography matrices to achieve image stitching and fusion. However, this method directly extracts features from images and then stitches them, which is not suitable for SLAM technology and is difficult to directly match SLAM map format files. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of large-scale mapping in the above-mentioned prior art and to provide a SLAM map stitching method, device, equipment and medium based on deep network. Only two PNG pictures or SLAM map files need to be imported to complete the map stitching. The exported map can be directly used by the robot, reducing the cost of mapping and improving the efficiency of mapping.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] According to a first aspect of the present invention, a SLAM map splicing method based on a deep network is provided, the method comprising the following steps:

[0009] Get a pair of slam map images to be stitched;

[0010] Perform matrix processing on the slam map image and convert it into a single-channel binary image matrix;

[0011] Obtain user instructions to determine whether to select a matching area. If so, use the single-channel binary image matrix corresponding to the matching area selected by the user as the input of the deep network. Otherwise, directly use the complete single-channel binary image matrix as the input of the deep network.

[0012] Use deep network to extract matrix features and output the homography transformation matrix between two rectangular input images;

[0013] According to the homography transformation matrix, the position of each pixel in the first single-channel binary image matrix in the second single-channel binary image matrix is ​​determined to complete the image stitching.

[0014] As a preferred technical solution, the matrix processing of the SLAM map image is specifically as follows: the raster map matrix with a pixel value range of 0-255 is converted into a binary matrix with a value range of 0 or 1 through binarization processing to obtain a single-channel binary image matrix.

[0015] As a preferred technical solution, the deep network includes an input layer, a feature extraction layer and an output layer connected in sequence, wherein the input layer concatenates two input image matrices to obtain an input tensor, the feature extraction layer performs feature extraction on the input tensor to obtain the x- and y-direction offsets of the four vertices between the rectangular input images, and outputs them through the output layer as a vector of length 8, i.e., the predicted values ​​of the coordinate offsets of the four vertices, representing the parameters of the homography transformation matrix.

[0016] As a preferred technical solution, the input layer concatenates a pair of input image matrices of size H×W in the channel dimension to form a 2×H×W tensor, where H represents the image height and W represents the image width.

[0017] As a preferred technical solution, the feature extraction layer includes eight convolutional layers, three pooling layers and two fully connected layers, wherein:

[0018] The first convolutional layer convolves the input 2×H×W tensor and outputs a 64×H×W feature map.

[0019] The second convolutional layer applies a convolution operation to the output of the first convolutional layer and outputs a 64×H×W feature map;

[0020] The third convolutional layer further extracts features from the output of the second convolutional layer and outputs a 128×H×W feature map;

[0021] The fourth convolutional layer further extracts features from the output of the third convolutional layer and outputs a 128×H×W feature map;

[0022] The fifth convolutional layer maps the features output by the fourth convolutional layer to 256×H×W dimensions;

[0023] The sixth convolutional layer further extracts features from the output of the fifth convolutional layer and outputs a 256×H×W feature map;

[0024] The seventh convolutional layer further extracts features from the output of the sixth convolutional layer and outputs a 512×H×W feature map;

[0025] The eighth convolutional layer outputs a 512×H / 2×W / 2 feature map through convolution and pooling operations;

[0026] A pooling layer is added after the second, fourth, and sixth convolutional layers to reduce the spatial dimension of the feature map;

[0027] The feature map output by the eighth convolutional layer is flattened and mapped to a 1024-dimensional hidden layer through the first fully connected layer, and a nonlinear transformation is performed using the ReLU activation function;

[0028] The second fully connected layer maps the output of the hidden layer to the 8-dimensional output layer.

[0029] As a preferred technical solution, the deep network adopts the L2 loss function, which is used to calculate the error between the predicted four vertex coordinates and the real vertex coordinates. The predicted four vertex coordinates are the coordinate offset prediction values ​​of the four vertices output by the deep network, and the real vertex coordinates are the pre-made 4 offset gt data.

[0030] As a preferred technical solution, eight equations are provided to solve the homography transformation matrix H according to the coordinate offset prediction values ​​of the four vertices output by the deep network:

[0031]

[0032] in, represents the coordinates of the i-th vertex in the first single-channel binary image matrix, represents the coordinates of the i-th vertex in the second single-channel binary image matrix, Represents the coordinate offset prediction value of the i-th vertex output by the deep network.

[0033] According to a second aspect of the present invention, a SLAM map splicing device based on a deep network is provided, comprising:

[0034] Image acquisition module: used to obtain a pair of SLAM map images to be stitched;

[0035] Conversion processing module: used to perform matrix processing on the SLAM map image and convert it into a single-channel binary image matrix;

[0036] Region selection module: used to obtain user instructions to determine whether to select a matching region. If so, the single-channel binary image matrix corresponding to the matching region selected by the user is used as the input of the deep network. Otherwise, the complete single-channel binary image matrix is ​​directly used as the input of the deep network.

[0037] Homography transformation matrix solving module: used to extract matrix features using a deep network and output the homography transformation matrix between two rectangular input images;

[0038] Image stitching module: used to determine the position of each pixel in the first single-channel binary image matrix in the second single-channel binary image matrix according to the homography transformation matrix, and complete image stitching.

[0039] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0040] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described when executed by a processor.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The present invention uses a deep network to process maps and understand the global features of the image instead of using local feature point features. This effectively solves the problem of stitching failure caused by sparse feature points and large map differences in the case of a relatively empty map.

[0043] (2) The present invention directly obtains the homography transformation matrix between the two maps through a deep network, rather than using the affine transformation matrix calculated by the traditional algorithm. This can solve the perspective projection problem between the two maps and allow png images or slam map files to be directly used for splicing without conversion. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 A schematic diagram of a slam map image to be spliced ​​in an embodiment;

[0046] Figure 3 A schematic diagram of matching area selection in an embodiment;

[0047] Figure 4 It is a schematic diagram of the deep network structure of the present invention;

[0048] Figure 5 A schematic diagram of a map stitching result in an embodiment. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The present invention designs a map stitching method for SLAM maps, which is an offline map stitching method that relies on deep learning and is mainly used to stitch and crop maps constructed by robots. It supports the input of common SLAM map format files and maps in the form of PNG images, and realizes a high-precision, highly operable and fast stitching tool.

[0052] Specifically, Figure 1 As shown, the method comprises the following steps:

[0053] S1, obtain a pair of slam map images to be stitched, such as Figure 2 shown.

[0054] S2, matrix processing is performed on the slam map image, and the raster map matrix with pixel value range between 0-255 is converted into a binary matrix with value range of 0 or 1 through binarization processing, and converted into a single-channel binary image matrix.

[0055] S3, obtain the user instruction to determine whether to select the matching area. If so, use the single-channel binary image matrix corresponding to the matching area selected by the user as the input of the deep network, such as Figure 3 As shown, otherwise, the complete single-channel binary image matrix is ​​directly used as the input of the deep network.

[0056] S4, uses a deep network to extract matrix features and outputs the homography transformation matrix between the two rectangular input images.

[0057] Traditional image registration mainly uses feature-based methods. These methods have three steps: key point detection and feature description, feature matching, and image transformation. Simply put, we select points of interest in two images, associate the reference image with the equivalent points of interest in the sensed image, and then transform the sensed image to align the two images. In the present invention, a convolutional neural network is used to directly learn the geometric transformation to align the two images, rather than just being limited to feature extraction.

[0058] The present invention uses the HomographyNe regression network to learn the homography of two images and outputs the transformation matrix in an end-to-end manner. Compared with the traditional image registration steps mentioned above, the first two steps are omitted.

[0059] like Figure 4 As shown, the deep network includes an input layer, a feature extraction layer, and an output layer connected in sequence.

[0060] 1) Input layer

[0061] The input layer concatenates a pair of H×W image matrices in the channel dimension to form a 2×H×W tensor, where H represents the image height and W represents the image width.

[0062] 2) The feature extraction layer extracts features from the input tensor and obtains the x- and y-direction offsets of the four vertices of the rectangular input image.

[0063] The feature extraction layer includes eight convolutional layers, three pooling layers, and two fully connected layers.

[0064] Convolutional Layer:

[0065] Conv1: The input 2×H×W tensor is convolved to output a 64×H×W feature map; the convolution kernel size used is usually 3×3, and appropriate padding is used to keep the image size unchanged.

[0066] Conv2: Apply another convolution operation to the output of Conv1, outputting a 64×H×W feature map.

[0067] Conv3: further extracts features from the output of Conv2 and outputs a 128×H×W feature map.

[0068] Conv4: further extracts features from the output of Conv3 and outputs a 128×H×W feature map.

[0069] Conv5: Maps the features output by Conv4 to 256×H×W dimensions.

[0070] Conv6: further extracts features from the output of Conv5 and outputs a 256×H×W feature map.

[0071] Conv7: further extracts features from the output of Conv6 and outputs a 512×H×W feature map.

[0072] Conv8: Outputs a 512×H / 2×W / 2 feature map through convolution and pooling operations, where the spatial dimension is reduced through pooling operations (such as maximum pooling).

[0073] Pooling layer:

[0074] A pooling layer is added after each of Conv2, Conv4 and Conv6 to reduce the spatial dimension of the feature map. In this embodiment, a 2×2 window is applied to the feature map using Max Pooling to reduce its width and height.

[0075] Fully connected layer:

[0076] Fully connected layer 1: The feature map output by Conv8 is flattened and mapped to a 1024-dimensional hidden layer through fully connected layer 1, and a nonlinear transformation is performed using the ReLU activation function.

[0077] Fully connected layer 2: maps the output of the hidden layer to an 8-dimensional output layer. These 8 values ​​correspond to the parameters of the homography matrix (the x and y offsets of the four points).

[0078] 3) Output layer

[0079] The output layer outputs a vector of length 8, which is the predicted value of the coordinate offset of the four vertices, representing the parameters of the homography transformation matrix.

[0080] In this embodiment, the deep network uses the L2 loss function, which is used to calculate the error between the predicted four vertex coordinates and the real vertex coordinates. The predicted four vertex coordinates are the coordinate offset prediction values ​​of the four vertices output by the deep network, and the real vertex coordinates are the pre-made four offset gt data. The loss function is expressed as:

[0081]

[0082] Among them, h i is the parameter output by the network, is the real parameter.

[0083] S5, determining the position of each pixel point in the first single-channel binary image matrix in the second single-channel binary image matrix according to the homography transformation matrix, and completing image stitching.

[0084] Suppose there is a point x on map a 1 and there is a point x on map b 2 , H represents the homography transformation matrix. Assuming that these two points represent the same point in three-dimensional space, the formula is satisfied:

[0085] x 2 =Hx 1

[0086] Expand to:

[0087]

[0088] Among them, (u 1 ,v 1 ), (u 2 ,v 2 ) are the coordinates of the points in map a and map b respectively.

[0089] Because the transformation is homogeneous coordinates, the homography transformation matrix H is independent of scale and has 8 degrees of freedom. 33 To normalize.

[0090]

[0091] Therefore, H has eight unknowns and eight equations need to be solved. For a pair of solution points, the expansion form in homogeneous coordinates is as follows:

[0092]

[0093] It can be seen that one pair of solution points can get two solution equations, so four pairs of matching points are needed to provide eight equations for solution. According to the coordinate offset prediction values ​​of the four vertices output by the deep network, eight equations are provided to solve the homography transformation matrix H:

[0094]

[0095] in, represents the coordinates of the i-th vertex in the first single-channel binary image matrix, represents the coordinates of the i-th vertex in the second single-channel binary image matrix, Represents the coordinate offset prediction value of the i-th vertex output by the deep network.

[0096] Due to the advanced nature of this technical solution, it can be widely used in the fields of robot navigation, image processing, etc. First, for the field of robot navigation, this technical solution solves the problems of difficulty in building maps for large scenes, low precision and high cost, overcomes the problem that the traditional map splicing algorithm fails to build maps due to the sparse feature points of open scenes, and solves the problem that the failure rate of the traditional splicing algorithm is high when the map difference is too large, improves the robustness of the algorithm, reduces the cost and difficulty of building maps in a large range, and improves the accuracy of building maps, thereby improving the navigation ability of the robot, enabling it to navigate autonomously in a more complex environment. Secondly, for the field of image processing, this technical solution can realize accurate registration and splicing of images, and improve the robustness and precision of image processing. Finally, the algorithm can be retrained with data for the corresponding tasks, and can also be applied to other scenarios such as autonomous driving, virtual reality, and augmented reality. For example, map splicing between multiple autonomous driving vehicles can be realized, thereby achieving more accurate path planning and collaborative driving. Or it can achieve seamless connection between virtual scenes and real scenes, providing a more realistic immersive experience, etc. In general, this technical solution has broad market demand and good application prospects.

[0097] The above is an introduction to a method embodiment. The following is a further explanation of the solution of the present invention through an apparatus embodiment.

[0098] A SLAM map splicing device based on a deep network, comprising:

[0099] Image acquisition module: used to obtain a pair of SLAM map images to be stitched;

[0100] Conversion processing module: used to perform matrix processing on the SLAM map image and convert it into a single-channel binary image matrix;

[0101] Region selection module: used to obtain user instructions to determine whether to select a matching region. If so, the single-channel binary image matrix corresponding to the matching region selected by the user is used as the input of the deep network. Otherwise, the complete single-channel binary image matrix is ​​directly used as the input of the deep network.

[0102] Homography transformation matrix solving module: used to extract matrix features using a deep network and output the homography transformation matrix between two rectangular input images;

[0103] Image stitching module: used to determine the position of each pixel in the first single-channel binary image matrix in the second single-channel binary image matrix according to the homography transformation matrix, and complete image stitching.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0105] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0106] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0107] The processing unit performs the various methods and processes described above, such as methods S1 to S5. For example, in some embodiments, methods S1 to S5 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S5 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S5 in any other appropriate manner (e.g., by means of firmware).

[0108] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0109] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0111] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A SLAM map splicing method based on deep network, characterized in that: The method comprises the following steps: Get a pair of slam map images to be stitched; Perform matrix processing on the slam map image and convert it into a single-channel binary image matrix; Obtain user instructions to determine whether to select a matching area. If so, use the single-channel binary image matrix corresponding to the matching area selected by the user as the input of the deep network. Otherwise, directly use the complete single-channel binary image matrix as the input of the deep network. Use deep network to extract matrix features and output the homography transformation matrix between two rectangular input images; According to the homography transformation matrix, the position of each pixel in the first single-channel binary image matrix in the second single-channel binary image matrix is ​​determined to complete the image stitching.

2. According to the deep network-based SLAM map splicing method of claim 1, it is characterized in that: The matrix processing of the SLAM map image is specifically as follows: the grid map matrix with pixel value range between 0-255 is converted into a binary matrix with value range of 0 or 1 through binarization processing to obtain a single-channel binary image matrix.

3. A SLAM map splicing method based on deep network according to claim 1, characterized in that: The deep network includes an input layer, a feature extraction layer and an output layer connected in sequence, wherein the input layer concatenates two input image matrices to obtain an input tensor, the feature extraction layer performs feature extraction on the input tensor to obtain the x-direction and y-direction offsets of the four vertices between the rectangular input images, and outputs them through the output layer as a vector with a length of 8, i.e., the coordinate offset prediction values ​​of the four vertices, representing the parameters of the homography transformation matrix.

4. A SLAM map splicing method based on deep network according to claim 3, characterized in that: The input layer concatenates a pair of input image matrices of size H×W in the channel dimension to form a 2×H×W tensor, where H represents the image height and W represents the image width.

5. A SLAM map splicing method based on deep network according to claim 3, characterized in that: The feature extraction layer includes eight convolutional layers, three pooling layers and two fully connected layers, where: The first convolutional layer convolves the input 2×H×W tensor and outputs a 64×H×W feature map. The second convolutional layer applies a convolution operation to the output of the first convolutional layer and outputs a 64×H×W feature map; The third convolutional layer further extracts features from the output of the second convolutional layer and outputs a 128×H×W feature map; The fourth convolutional layer further extracts features from the output of the third convolutional layer and outputs a 128×H×W feature map; The fifth convolutional layer maps the features output by the fourth convolutional layer to 256×H×W dimensions; The sixth convolutional layer further extracts features from the output of the fifth convolutional layer and outputs a 256×H×W feature map; The seventh convolutional layer further extracts features from the output of the sixth convolutional layer and outputs a 512×H×W feature map; The eighth convolutional layer outputs a 512×H / 2×W / 2 feature map through convolution and pooling operations; A pooling layer is added after the second, fourth, and sixth convolutional layers to reduce the spatial dimension of the feature map; The feature map output by the eighth convolutional layer is flattened and mapped to a 1024-dimensional hidden layer through the first fully connected layer, and a nonlinear transformation is performed using the ReLU activation function; The second fully connected layer maps the output of the hidden layer to the 8-dimensional output layer.

6. A SLAM map splicing method based on deep network according to claim 1, characterized in that: The deep network adopts the L2 loss function, which is used to calculate the error between the predicted four vertex coordinates and the real vertex coordinates. The predicted four vertex coordinates are the coordinate offset prediction values ​​of the four vertices output by the deep network, and the real vertex coordinates are the pre-made 4 offset gt data.

7. A SLAM map splicing method based on deep network according to claim 1, characterized in that: According to the coordinate offset prediction values ​​of the four vertices output by the deep network, eight equations are provided to solve the homography transformation matrix H: in, represents the coordinates of the i-th vertex in the first single-channel binary image matrix, represents the coordinates of the i-th vertex in the second single-channel binary image matrix, Represents the coordinate offset prediction value of the i-th vertex output by the deep network.

8. A SLAM map splicing device based on deep network, characterized in that: include: Image acquisition module: used to obtain a pair of SLAM map images to be stitched; Conversion processing module: used to perform matrix processing on the SLAM map image and convert it into a single-channel binary image matrix; Region selection module: used to obtain user instructions to determine whether to select a matching region. If so, the single-channel binary image matrix corresponding to the matching region selected by the user is used as the input of the deep network. Otherwise, the complete single-channel binary image matrix is ​​directly used as the input of the deep network. Homography transformation matrix solving module: used to extract matrix features using a deep network and output the homography transformation matrix between two rectangular input images; Image stitching module: used to determine the position of each pixel in the first single-channel binary image matrix in the second single-channel binary image matrix according to the homography transformation matrix, and complete image stitching.

9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images