Loopback detection method and device based on visual SLAM, terminal equipment and storage medium

By using the GhostNet network model and an improved edge extraction algorithm, combined with grain warehouse address number recognition, the problem of misjudgment in visual SLAM loop closure detection in grain warehouse environments was solved, improving the accuracy and precision of loop closure detection and ensuring the accuracy of grain transport vehicle positioning and mapping.

CN118941631APending Publication Date: 2024-11-12HENAN UNIVERSITY OF TECHNOLOGY +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410990472.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In grain depot environments, the loop closure detection module of visual SLAM systems is prone to misclassifying different grain depot buildings as the same one, leading to a decrease in the accuracy of localization and mapping.

Method used

Image features are extracted using a pre-trained GhostNet network model, and dimensionality reduction is performed by combining principal component analysis and binarization algorithms. Hamming distance is used to determine location information, and grain warehouse door numbers are identified through image edge recognition and template matching techniques. A convolutional neural network is built for accurate number recognition, and traditional edge extraction algorithms are improved to enhance the accuracy of grain warehouse building recognition.

Benefits of technology

It effectively solves the problem of misjudgment of loopback detection in grain depot environments, improves the accuracy and precision of loopback detection, and ensures the accuracy of positioning and mapping of grain transport vehicles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941631A_ABST
    Figure CN118941631A_ABST
Patent Text Reader

Abstract

The invention discloses a loopback detection method and device based on visual SLAM (Simultaneous Localization and Mapping), terminal equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected image; determining a feature matrix corresponding to the to-be-detected image according to the to-be-detected image and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix according to the feature matrix; performing image edge identification on the to-be-detected image, and determining an object identifier in the to-be-detected image; according to the position information and the object identification, loopback judgment is carried out, the building corresponding to the object identification is determined, loopback detection in the grain depot environment is jointly completed by obtaining the object and the preset distance in the to-be-detected image and identifying the object identification in the image, the problem of loopback misjudgment is effectively solved, and the accuracy and precision of loopback detection are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of visual detection technology, and in particular, relates to a loop detection method, apparatus, terminal device and storage medium based on visual SLAM. Background Art

[0002] Grain logistics is the link that runs through the entire grain industry chain from production to consumption. Traditional grain depots have problems such as low efficiency in grain transfer, waste of human resources, and low level of intelligence. Strengthening the research and development of modern grain logistics is of great significance to improving the efficiency of grain circulation and increasing the country's emergency regulation and control capabilities for the grain market. The use of intelligent grain transfer vehicles and the introduction of Internet technology, integrated Beidou positioning, high-precision maps, multi-sensor information fusion and industrial cloud platforms in the grain depot transfer process can effectively improve the level of intelligence in the grain in and out operations. During the transfer process of intelligent grain transfer vehicles, unmanned and team-based information management is carried out throughout the process. The grain situation and vehicle information in the vehicle are directly related to the grain depot information cloud platform, realizing the true integration of modern information technology and intelligent logistics. In this context, the accuracy of positioning and mapping grain depots using intelligent grain transfer vehicles as carriers is particularly important.

[0003] As SLAM technology enters the robust era, the camera-based visual synchronous positioning and mapping method has developed rapidly in practical applications. The loop detection module is a very important module in the visual SLAM system. During the operation of the vehicle equipment, the loop detection module can identify the location that the equipment has reached. When the loop is determined to be generated, this information is sent to the back-end of the SLAM system for optimization, and the gaps between the same positions on the map are fitted to generate a trajectory consistent with the actual trajectory, thereby improving the accuracy of positioning and mapping.

[0004] The intelligent grain transfer vehicle operates in a grain storage environment, where there are a large number of similar grain storage buildings evenly distributed in the grain storage environment. The loop detection in the visual SLAM system is greatly hindered when operating in the grain storage environment. When using the traditional bag-of-words model for loop detection, different grain storage buildings are easily judged as the same grain storage building, resulting in incorrect fitting at the back end. How to improve the accuracy of grain storage building recognition is an urgent problem to be solved. Summary of the invention

[0005] The present application aims to provide a loop detection method, apparatus, terminal device and storage medium based on visual SLAM to solve the deficiencies in the prior art. The technical problem to be solved by the present application is achieved through the following technical solutions.

[0006] In a first aspect, an embodiment of the present application provides a loop detection method based on visual SLAM, the method comprising:

[0007] Acquire the image to be detected;

[0008] Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix according to the feature matrix;

[0009] Performing image edge recognition on the image to be detected to determine the object identification in the image to be detected;

[0010] A loop determination is performed according to the position information and the object identifier to obtain a loop determination result, and the loop determination result is used to update the historical data.

[0011] Optionally, determining a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix based on the feature matrix includes:

[0012] Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using a first sample image;

[0013] The principal component analysis method is used to reduce the dimension of the feature matrix to obtain a feature vector after the dimension reduction;

[0014] Using a binarization algorithm to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector;

[0015] The Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

[0016] Optionally, the adopting a binarization algorithm to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector includes:

[0017] Calculating the covariance matrix of the eigenvector after dimensionality reduction;

[0018] Performing singular value decomposition on the covariance matrix of the eigenvector to obtain a left singular matrix and a right singular matrix;

[0019] Processing the reduced-dimensional eigenvector according to the left singular matrix to obtain a processed matrix;

[0020] Calculate the mean of each row of the processed matrix;

[0021] The low-dimensional feature vector is determined according to the mean of each row.

[0022] Optionally, performing image edge recognition on the image to be detected to determine the object identification in the image to be detected includes:

[0023] Matching the image to be detected according to a pre-stored template library to determine the object contour of the image to be detected;

[0024] Using an edge extraction algorithm to extract the edge of the object contour to obtain edge information corresponding to the object contour;

[0025] According to the edge information, the object in the image to be detected is cropped to obtain a cropped image;

[0026] According to a pre-trained digital recognition model and the cropped image, the object identification in the cropped image is determined, wherein the pre-trained digital recognition model is obtained by training a convolutional neural network model using the second sample data.

[0027] Optionally, the extracting the edge of the object contour by using an edge extraction algorithm to obtain edge information corresponding to the object contour includes:

[0028] Calculate the direction vector of each pixel in the image to be detected;

[0029] Calculating the dot product of the direction vector of the pixel point in the template in the template library and the direction vector of the pixel point in the image to be detected;

[0030] According to the dot product, determining whether a pixel point in the image to be detected matches a pixel point in the template;

[0031] According to the matching result of the pixel points, edge information corresponding to the contour of the object is determined.

[0032] Optionally, the method further comprises:

[0033] When the pixel gradient value of a pixel point in the image to be detected is greater than a high threshold, the pixel point is determined to be an edge point;

[0034] When the pixel gradient value of a pixel point in the image to be detected is less than a low threshold, the pixel point is determined to be a normal pixel point;

[0035] If the pixel gradient value of a pixel point in the image to be detected is between the upper threshold and the lower threshold, then it is determined whether the pixel point is an edge point based on the pixel gradient values ​​of the neighborhood surrounding the pixel point.

[0036] In a second aspect, an embodiment of the present application provides a loop detection device based on visual SLAM, the device comprising:

[0037] An acquisition module, used for acquiring an image to be detected;

[0038] A first determination module is used to determine a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix based on the feature matrix;

[0039] A second determination module is used to perform image edge recognition on the image to be detected and determine the object identification in the image to be detected;

[0040] The loop module is used to perform loop determination according to the position information and the object identifier to obtain a loop determination result, and the loop determination result is used to update the historical data.

[0041] Optionally, the first determining module is used to:

[0042] Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using a first sample image;

[0043] The principal component analysis method is used to reduce the dimension of the feature matrix to obtain a feature vector after the dimension reduction;

[0044] Using a binarization algorithm to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector;

[0045] The Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

[0046] Optionally, the first determining module is used to:

[0047] Calculating the covariance matrix of the eigenvector after dimensionality reduction;

[0048] Performing singular value decomposition on the covariance matrix of the eigenvector to obtain a left singular matrix and a right singular matrix;

[0049] Processing the reduced-dimensional eigenvector according to the left singular matrix to obtain a processed matrix;

[0050] Calculate the mean of each row of the processed matrix;

[0051] The low-dimensional feature vector is determined according to the mean of each row.

[0052] Optionally, the second determining module is used to:

[0053] Matching the image to be detected according to a pre-stored template library to determine the object contour of the image to be detected;

[0054] Using an edge extraction algorithm to extract the edge of the object contour to obtain edge information corresponding to the object contour;

[0055] According to the edge information, the object in the image to be detected is cropped to obtain a cropped image;

[0056] According to a pre-trained digital recognition model and the cropped image, the object identification in the cropped image is determined, wherein the pre-trained digital recognition model is obtained by training a convolutional neural network model using the second sample data.

[0057] Optionally, the second determining module is used to:

[0058] Calculate the direction vector of each pixel in the image to be detected;

[0059] Calculating the dot product of the direction vector of the pixel point in the template in the template library and the direction vector of the pixel point in the image to be detected;

[0060] According to the dot product, determining whether a pixel point in the image to be detected matches a pixel point in the template;

[0061] According to the matching result of the pixel points, edge information corresponding to the contour of the object is determined.

[0062] Optionally, the second determining module is used to:

[0063] When the pixel gradient value of a pixel point in the image to be detected is greater than a high threshold, the pixel point is determined to be an edge point;

[0064] When the pixel gradient value of a pixel point in the image to be detected is less than a low threshold, the pixel point is determined to be a normal pixel point;

[0065] If the pixel gradient value of a pixel point in the image to be detected is between the upper threshold and the lower threshold, then it is determined whether the pixel point is an edge point based on the pixel gradient values ​​of the neighborhood surrounding the pixel point.

[0066] In a third aspect, an embodiment of the present application provides a terminal device, including: at least one processor and a memory;

[0067] The memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the visual SLAM-based loop detection method provided in the first aspect.

[0068] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the visual SLAM-based loop detection method provided in the first aspect is implemented.

[0069] The embodiments of the present application include the following advantages:

[0070] The visual SLAM-based loop detection method, apparatus, terminal device and storage medium provided in the embodiments of the present application obtain an image to be detected; determine a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix based on the feature matrix; perform image edge recognition on the image to be detected and determine the object identification in the image to be detected; perform loop determination based on the position information and the object identification and determine the building corresponding to the object identification, and complete loop detection in a grain storage environment by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0072] Figure 1 This is a flow chart of a loop detection method based on visual SLAM in one embodiment of the present application;

[0073] Figure 2 This is a flowchart of another loop detection method based on visual SLAM in an embodiment of the present application;

[0074] Figure 3 This is a schematic diagram of the bottleneck structure in the GhostNet network in one embodiment of the present application;

[0075] Figure 4 This is a schematic diagram of image template matching in an embodiment of the present application;

[0076] Figure 5 An image template library constructed in an embodiment of the present application;

[0077] Figure 6 is a template profile assumed in an embodiment of the present application;

[0078] Figure 7This is a flow chart of adaptive median filtering in one embodiment of the present application;

[0079] Figure 8 This is a comparison of edge extraction algorithms in an embodiment of the present application;

[0080] Fig. 9 It is a structural block diagram of an embodiment of a loop detection device based on visual SLAM of the present application;

[0081] Fig.10 It is a structural diagram of a terminal device of the present application. DETAILED DESCRIPTION

[0082] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0083] An embodiment of the present application provides a loop detection method based on visual SLAM, which is used to perform loop detection on a grain storage environment. The execution subject of this embodiment is a loop detection device based on visual SLAM, which is set on a terminal device, for example, the terminal device at least includes a computer terminal, etc.

[0084] Reference Figure 1 , shows a flowchart of a loop detection method embodiment based on visual SLAM of the present application, which may specifically include the following steps:

[0085] S101, obtaining an image to be detected;

[0086] Specifically, the terminal device obtains the image to be detected in the grain storage environment, wherein the image to be detected can be a video or a picture.

[0087] S102, determining a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix according to the feature matrix;

[0088] Specifically, the terminal device enters the current frame in the collected image to be detected into the feature extraction network model, outputs the feature matrix corresponding to the current frame, uses principal component analysis (PCA) and binarization to process the feature matrix corresponding to the current frame, and converts the feature matrix corresponding to the current frame into a low-dimensional binary vector, and then uses Hamming distance comparison to obtain the position information corresponding to the feature matrix.

[0089] S103, performing image edge recognition on the image to be detected to determine the object identification in the image to be detected;

[0090] Specifically, the terminal device performs image edge detection on the current frame in the image to be detected, uses image template matching technology to roughly locate the object identifier in the current frame in the image to be detected, crops the potential area, and classifies and identifies the cropped image in the CNN network model, where the object identifier can be a number, letter or.

[0091] S104: Perform loop determination based on the position information and the object identifier to obtain a loop determination result, which is used to update the historical data.

[0092] Specifically, the terminal device performs loop determination based on the obtained position information and object identification to obtain a loop determination result, and the loop determination result is used to update the historical data.

[0093] like Figure 2 As shown, an embodiment of the present application provides a visual SLAM loop detection system for an intelligent grain transfer vehicle, including: two branches of the overall architecture of the grain depot loop detection system: Branch one is for the current frame to enter the GhostNet network, after the feature matrix output, principal component analysis (PCA) and binarization are used to process the features of the image output, and the image feature vector is converted into a low-dimensional binary vector, and then the Hamming distance is used for comparison. Branch two first performs image edge detection on the current frame, and the image template matching technology roughly locates the numbers in the image, and the potential area is cropped and sent to CNN for digital classification. If there are numbers, a loop judgment is performed with branch one. The two are combined to complete the loop detection in the grain depot environment. The present invention can effectively solve the problem of loop misjudgment and improve the accuracy and precision of loop detection.

[0094] In the embodiment of the present application, the pre-trained GhostNet network is used as the basis for transfer learning of the grain depot data set. One layer is used as the result of deep-level feature extraction of the image, and the data is processed using operations such as principal component analysis, dimensionality reduction, and binarization. In view of the fact that a large number of repeated buildings in the grain depot scene are prone to loop misjudgment, a branch of granary house number recognition is introduced, the traditional edge extraction algorithm is improved, and the contour-based template matching is used for coarse positioning of the digital part. A convolutional neural network is built for accurate digital recognition. The two are combined to complete loop detection in the grain depot environment. This allows the new technology to be applied to grain depots.

[0095] The visual SLAM-based loop detection method provided in the embodiment of the present application obtains an image to be detected; determines a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determines position information corresponding to the feature matrix based on the feature matrix; performs image edge recognition on the image to be detected and determines the object identification in the image to be detected; performs loop determination based on the position information and the object identification and determines the building corresponding to the object identification, and completes loop detection in a grain storage environment by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection.

[0096] Another embodiment of the present application further supplements the loop detection method based on visual SLAM provided in the above embodiment.

[0097] Optionally, determining a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix according to the feature matrix, including:

[0098] Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using the first sample image;

[0099] The principal component analysis method is used to reduce the dimension of the feature matrix and obtain the feature vector after dimension reduction;

[0100] The binarization algorithm is used to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector;

[0101] Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

[0102] Specifically, in the embodiment of the present application, CNN is used to extract deep-level features of the image. The Ghost module contained in the GhostNet network replaces the standard convolution in the traditional convolutional neural network, and uses fewer convolution operations to generate more feature maps; the network migration training uses the GhostNet model trained on the ImageNet public dataset as the initialization weight of the training network, and constructs a grain depot-specific dataset for migration training, so that it can accurately classify objects in the grain depot; feature similarity measurement uses the output of the FC8 layer in the GhostNet network as the feature of the image, and uses principal component analysis (PCA) and binarization to process the features of the image output to improve the representation ability of image features.

[0103] That is, feature extraction of the GhostNet network, including:

[0104] The Ghost module included in the GhostNet network replaces the standard convolution in the traditional convolutional neural network, including: first using 1×1 convolution to generate some feature maps, then performing linear operations on these feature maps to generate more feature maps, and finally merging the two feature maps, adding output channels, and achieving the same output as ordinary convolution. In this way, the network using the Ghost module can use fewer parameters than conventional convolutional neural networks to generate the same number of feature maps, forming a lightweight network.

[0105] In the GhostNet network, the Ghost module is used to build the bottleneck structure in the GhostNet network. According to the different step sizes, it can be divided into the following Figure 3 The two bottleneck structures shown in the figure both have two Ghost modules. The bottleneck structure draws on the residual structure in the ResNet network. After the input comes in, it will pass through two Ghost modules to extract features of the input data. In the branch part, the input and the output processed by the two Ghost modules are directly combined and output.

[0106] The GhostNet network used reaches the first convolution layer after the image input comes in. This layer is a standard convolution layer with 16 convolution kernels, followed by a series of bottleneck structures including the Ghost module. The end of the network is the average pooling layer, convolution layer and fully connected layer, which are responsible for converting the feature map into the final classification.

[0107] The overall network structure is shown in Table 1.

[0108] Table 1 GhostNet network overall architecture diagram

[0109]

[0110] Among them, the migration training of the network includes: using the GhostNet model trained on the ImageNet public dataset as the initialization weights of the training network.

[0111] For grain depots in actual scenes, a grain depot scene dataset containing 1,180 images was generated using a monocular acquisition device, and the training set and test set were divided into 5:1. The dataset labels include shallow round silos, flat silos, silos, grain scrapers, bucket elevators, fans, scraper conveyors, belt conveyors, screw conveyors, mobile grain suction machines, cleaning screens, and weighing scales in the grain depot. Among them, shallow round silos, square silos, and silos are the main buildings. These mechanical equipment are generally only used in grain depot environments and have high recognition.

[0112] The size and number of images in the grain depot dataset are counted to better reflect the situation of the constructed grain depot dataset. The dataset contains a large number of square warehouses, shallow round warehouses and silos. These three types of grain warehouse buildings are extremely important targets in grain depots and are an important basis for identifying images in the loop detection system.

[0113] Use transfer learning to train the pre-trained GhostNet network. Fine-tune the last layer of the training model, the fully connected classification layer, and modify the output of the fully connected layer so that the output of the fully connected layer corresponds to the number of categories in the current dataset. During the training process, freeze the other layers and only fine-tune the parameters of the last fully connected layer.

[0114] Optionally, a binarization algorithm is used to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector, including:

[0115] Calculate the covariance matrix of the reduced eigenvector;

[0116] Perform singular value decomposition on the covariance matrix of the eigenvector to obtain a left singular matrix and a right singular matrix;

[0117] The eigenvector after dimension reduction is processed according to the left singular matrix to obtain a processed matrix;

[0118] Calculate the mean of each row of the processed matrix;

[0119] According to the mean of each row, a low-dimensional feature vector is determined.

[0120] Specifically, in the embodiment of the present application, it is necessary to measure the feature similarity, including:

[0121] The main function of the convolutional neural network is to classify and retrieve images, and the final output is not the image feature. From the overall structure of the GhostNet network, the last fully connected layer is the output layer of an image classification, so it is not suitable as a feature extraction layer. The features extracted by the layers too far forward are often rough or not deep enough and cannot represent the global image. Therefore, the output of the FC8 layer in the GhostNet network is used as the image feature.

[0122] The principal component analysis (PCA) and binarization are used to process the features of the image output to improve the representation ability of the image features.

[0123] After the image is input into the GhostNet network, the output of the FC8 layer is extracted as the deep features of the image. These features are high-dimensional vectors with a dimension of 1280. If the total number of images in the image library is m, and the dimension of the feature vector that can be extracted for each single image is n, then the sample matrix can be expressed as:

[0124] X=[X 1 ,X 2 ,...,X m ] T ∈R m×n

[0125] The vector of Xi is represented as:

[0126] X i =[x i1 ,x i2 ,...,x in ] T ∈R n ,i=1,2,...,m

[0127] The steps of data processing are as follows:

[0128] 1) Calculate the covariance matrix Σ of the sample matrix, which is calculated as follows:

[0129]

[0130] 2) Perform singular value decomposition (SVD) on the covariance matrix Σ.

[0131] The calculation method is

[0132] ∑=USV T ;

[0133] The U and V unitary matrices in the formula are left singular matrices and right singular matrices.

[0134] 3) Use the left singular matrix U to decorrelate the sample matrix. The calculation method is:

[0135] X r =U T X T

[0136] 4) Calculate the mean of each row of the sample matrix after dimensionality reduction. The calculation method is

[0137]

[0138] 5) Binarize the sample matrix. The calculation method is as follows

[0139]

[0140] After the above processing, the image feature vector is converted into a low-dimensional binary vector, and the Hamming distance can be used for subsequent similarity judgment.

[0141] Optionally, performing image edge recognition on the image to be detected to determine the object identification in the image to be detected includes:

[0142] According to the pre-stored template library, the image to be detected is matched to determine the object contour of the image to be detected;

[0143] An edge extraction algorithm is used to extract the edge of the object contour to obtain edge information corresponding to the object contour;

[0144] According to the edge information, the object in the image to be detected is cropped to obtain a cropped image;

[0145] According to a pre-trained digital recognition model and the cropped image, the object identification in the cropped image is determined, wherein the pre-trained digital recognition model is obtained by training a convolutional neural network model using the second sample data.

[0146] Specifically, the recognition of the granary numbers can be divided into two steps. The first step is to roughly locate the numbers and cut out the parts that may be numbers in the image with complex background. The second step is to input the image cut out in the first step into the built convolutional neural network for accurate number recognition to complete the recognition of the granary number.

[0147] The rough positioning of digital parts uses the template matching technology in image processing technology. Template matching technology first establishes a template library, which stores the content to be identified. The templates in the template library are used to traverse the input image, and the specific area in the image is identified by searching for the area matching the given template in the target image. The algorithm is as follows: Figure 4 As shown in the figure, the template matching based on the contour first uses the edge extraction algorithm to extract the edge in the image and process the contour of the template image; the template image is used to move rightward or downward from the upper left corner of the input image until the target position is traversed. In each of the above sliding, the difference between the template and the area in the image is calculated, and when the difference is less than a certain threshold, the template matching is determined to be successful.

[0148] A total of 9 image template libraries with sizes of 300 pixels × 300 pixels are constructed, from 1 to 9. When performing template matching, an image can be selected from the template library to match the input image. The template library images are as follows: Figure 5 shown.

[0149] The embodiment of the present application provides an improved edge extraction algorithm. The Canny algorithm calculates the gradient strength and direction of each pixel in the image, eliminates the stray effect caused by edge detection through non-maximum suppression, and finally combines dual threshold detection to determine the real and potential edges to complete edge detection; build CNN for digital classification, and after using digital templates to roughly locate the digital part in the image, mark the part of the image that may be digital and crop it, so that most of the cropped image is digital. The cropped part is input into the convolutional neural network for specific classification.

[0150] The pre-trained digital recognition model in the embodiment of the present application is obtained by training the convolutional neural network model using the second sample data, and specifically includes:

[0151] Convolutional neural network for recognizing numbers is built. Then the ReLU activation function is used to introduce nonlinear factors to the convolutional neural network, improve the fitting ability of the neural network, and enhance the expression ability of the convolutional network. It is followed by a pooling layer, which performs a pooling operation on the output of the activation function layer, reducing the image size to 7×7 pixels, and the number of channels remains unchanged. The next part of the network operates similarly to the previous part. The output passes through convolution layer 2, activation function, etc. and enters the end of the network. Two fully connected layers form the end of the network. Fully connected layer 1 has 1024 neurons, each of which is connected to all neurons in the previous layer for feature extraction and classification. Fully connected layer 2 has 10 neurons, which are used to output the classification results, and each neuron is connected to all neurons in the previous layer.

[0152] Use the SVHN (Street View House Number) training data set to train the constructed network so that it can complete the task of classifying numbers.

[0153] For example, a convolutional neural network for recognizing numbers is built. The overall architecture of the convolutional neural network is shown in Table 2. The input size of the input layer is 28×28×1, which means that the image input to the network is a 28×28 pixel, single-channel grayscale image. The convolution kernel used in the first convolution layer is 5×5×1, with a step size of 1 and no padding. The convolution layer has 32 convolution kernels, and the image size becomes 14×14×32 after the operation of the first convolution layer. The ReLU activation function is then used to introduce nonlinear factors to the convolutional neural network, improve the fitting ability of the neural network, and enhance the expression ability of the convolutional network. A pooling layer is connected to the back, with a pooling kernel size of 2×2 and a step size of 2. This layer performs a pooling operation on the output of the activation function layer, reducing the image size to 7×7 pixels, and the number of channels remains unchanged. The next part of the network is similar to the previous part. The output passes through convolution layer 2, activation function, etc. and enters the end of the network. Two fully connected layers form the end of the network. Fully connected layer 1 has 1024 neurons, each neuron is connected to all neurons in the previous layer, and is used to extract features and perform classification. Fully connected layer 2 has 10 neurons, which are used to output the classification results, and each neuron is connected to all neurons in the previous layer.

[0154] Table 2 Convolutional neural network structure

[0155]

[0156] After the network is built, it needs to be trained so that it can complete the task of classifying numbers. The SVHN (Street View House Number) dataset is used. The sizes, directions, and positions of the numbers contained in the dataset are all different, which helps the model learn to recognize the shape, size, and direction of numbers and perform better in more complex tasks.

[0157] After the loop detection module for the grain depot environment provided in the embodiment of the present application is constructed, the two branches use the GhostNet lightweight convolutional neural network and the self-built convolutional neural network to achieve different tasks. The repeated buildings in the grain depot cause great trouble for the loop detection problem of visual SLAM in the grain depot environment, and it is very easy to transmit wrong information to the SLAM backend. The pre-trained GhostNet network is used as the basis for transfer learning of the grain depot data set. One layer is used as the result of deep-level feature extraction of the image, and the principal component analysis is used to reduce the dimension and binarize the data. In view of the fact that a large number of repeated buildings in the grain depot scene are prone to loop misjudgment, a branch of granary house number recognition is introduced, the traditional edge extraction algorithm is improved, and the contour-based template matching is used for coarse positioning of the digital part. A convolutional neural network is built for accurate digital recognition. The two are combined to complete the loop detection in the grain depot environment.

[0158] Optionally, edge extraction algorithm is used to extract the edge of the object contour to obtain edge information corresponding to the object contour, including:

[0159] Calculate the direction vector of each pixel in the image to be detected;

[0160] Calculate the dot product between the direction vector of the pixel point in the template in the template library and the direction vector of the pixel point in the image to be detected;

[0161] According to the dot product, determine whether the pixel points in the image to be detected match the pixel points in the template;

[0162] According to the matching results of the pixel points, the edge information corresponding to the object contour is determined.

[0163] Specifically, the contour is not easily disturbed by the outside world during the template matching process, and has good robustness, which is suitable for the granary scene. At the same time, multiple templates can be matched at the same time during the matching process, which has good performance in terms of time performance. In the process of use, the edge extraction algorithm is first used to extract the edge in the image. The contour of the template image is processed, such as Figure 6 As shown in the figure, assuming that the contour extracted by the template is a hexagonal contour, the coordinate system is established with the centroid of the contour as the origin, then the coordinates of point p on the contour can be expressed as (x, y), and the direction vector of the point on the contour is t. The direction vector is obtained from the edge extraction algorithm. The coordinates of the point and the direction vector (p, t) can be combined to represent all the points on the contour and then the entire contour. In actual use, the contour may rotate. In order to improve the robustness of the algorithm, templates with different directions are usually established.

[0164] After the template operation is completed, the image can be matched based on the template. After edge extraction, there are a large number of points in the image. The direction vector e(v,w) of the point in the image is calculated, and the corresponding dot product is calculated between the direction vector in the template and the direction vector in the image to be matched, and then the similarity between a certain area in the image to be matched and the template is determined. In order to eliminate the interference of other factors, the direction vector of the contour point is normalized when used. The calculation formula is as follows. The closer the score is to 1, the higher the similarity between the part of the image to be matched and the template.

[0165]

[0166] After the template operation is completed, the image can be matched based on the template. After edge extraction, there are a large number of points in the image. The direction vector e(v,w) of the point in the image is calculated, and the corresponding dot product is calculated between the direction vector in the template and the direction vector in the image to be matched, and then the similarity between a certain area in the image to be matched and the template is determined.

[0167] Optionally, the method further comprises:

[0168] When the pixel gradient value of a pixel point in the image to be detected is greater than a high threshold, the pixel point is determined to be an edge point;

[0169] When the pixel gradient value of a pixel point in the image to be detected is less than a low threshold, the pixel point is determined to be a normal pixel point;

[0170] If the pixel gradient value of a pixel in the image to be detected is between the high threshold and the low threshold, then the pixel is judged to be an edge point based on the pixel gradient values ​​of the neighborhood pixels around the pixel.

[0171] Specifically, the embodiment of the present application improves the traditional Canny algorithm: a hybrid filter of a Gaussian filter and an adaptive median filter is used to replace the original Gaussian filter, and an adaptive high and low threshold solution is used to replace the original manually selected high and low thresholds.

[0172] The Canny algorithm is used to extract the edge of the image. The Canny algorithm calculates the gradient strength and direction of each pixel in the image, eliminates the stray effects caused by edge detection through non-maximum suppression, and finally combines dual threshold detection to determine the real and potential edges to complete the edge detection.

[0173] In the final stage of the Canny edge extraction algorithm, the high and low thresholds that are manually input are used to judge the pixels in the image, resulting in poor adaptability to the image. Therefore, the traditional Canny algorithm is improved from two aspects: using a hybrid filter of a Gaussian filter and an adaptive median filter to replace the original Gaussian filter, and selecting an adaptive high and low threshold solution to replace the original manually selected high and low thresholds.

[0174] (1) Hybrid filtering of Gaussian filter combined with adaptive median filter

[0175] After the image enters the system, it is first processed by a hybrid filter. Among them, the Gaussian filter is designed to effectively remove Gaussian noise in the image, while the adaptive median filter mainly processes salt and pepper noise. In addition, the adaptive termination filter is a special median filter that can dynamically adjust the size or shape of the filter window according to the statistical characteristics of the local area of ​​the image.

[0176] Ordinary median filtering is a nonlinear filtering method. It sorts the pixel values ​​in the window and takes the median as the center value. To ensure the convenience of median selection, the number of pixels in the filter window should be an odd number, so the window is mostly 3×3 or 5×5. Figure 7 As shown, Figure 7This is a flowchart of adaptive median filtering in an embodiment of the present application. A 3×3 filter window is used, which contains 9 pixels. The median filter arranges the 9 pixels according to their size and selects the median of the 9 pixels as the central pixel value of the 3×3 filter window.

[0177] Adaptive median filtering has three advantages over traditional median filtering. First, adaptive median filtering has better image noise processing capabilities, can better handle salt and pepper noise in images, and can better retain image details than traditional median filtering. Secondly, adaptive median filtering can dynamically adjust the filter window. The filter window of traditional median filtering is fixed, and can only use 3×3, 5×5 or 7×7 size windows to process images. It cannot make corresponding processing according to the actual noise density in the image. When the noise density of a part of the image is high, the filter can use a larger filter window to process that part. When the noise density of a part of the image is low, a smaller window can be used to retain more image details. Adaptive median filtering reflects better flexibility and pertinence. The core of the adaptive filter is to dynamically adjust the filter window. The principle is as follows:

[0178] 1) Given an initial filter window size of m×m, a final filter window size of n×n, and a grayscale value of Z at coordinate xy in the image xy ;

[0179] 2) Count the maximum value Z of the pixel grayscale value in the current window max , minimum value Z min , median Z med , where Z med -Z min Denoted as a 1 , Z med -Z max Denoted as a 2 ;

[0180] 3) Determine a in step 2) 1 and a 2 The value of 1 >0 and a 2 <0, go to step 4), otherwise change the size of the window, increase the size by 2, and re-execute the above process until the termination condition is reached, and output Z med ;

[0181] 4) Z xy -Z min Denoted as b 1 , Z xy -Z max Denoted as b 2 , judge b1 and b 2 If b is satisfied, 1 >0 and b 2 <0, the filter output value is Z xy , otherwise output Z med .

[0182] The flowchart of adaptive median filtering is as follows Figure 7 shown.

[0183] The change of the filter window size in the process is the core of the adaptive median filter. The judgment is mainly about salt and pepper noise. In the third step, it is judged whether the output is salt and pepper noise. When the output value is the maximum or minimum value in the filter window, it may be salt and pepper noise. It is necessary to increase the size of the filter window and calculate again. Otherwise, it goes to the fourth step. The final output determines whether it is salt and pepper noise. When the center pixel of the filter window is not salt and pepper noise, the current pixel value is output. If the output is salt and pepper noise, the median is assigned to the current filter pixel as the filter output value.

[0184] The filtering algorithm is tested using images with Gaussian noise and salt and pepper noise. The original image used is the digital number on the door frame of the square warehouse granary, which plays an important role in the system as a basis for judgment.

[0185] (2) Adaptive high and low thresholds

[0186] In the fourth step of the original canny edge detection algorithm, the high and low thresholds need to be set manually. When using the manually set high and low thresholds for edge detection, it cannot be adjusted according to the actual situation of the image. When the high threshold is too high, the points that should have been edge points will be demarcated as ordinary pixels. When the low threshold is too low, the points that were not edge points will be demarcated as edge points. This section selects appropriate high and low thresholds based on the actual situation of the image and applies them to the canny edge extraction algorithm. The calculation formula is shown below.

[0187]

[0188] Where x(i,j) is the grayscale value of the pixel at position i, j.

[0189] The threshold T can be calculated by the above formula. At this time, the threshold T is the high threshold in the edge extraction algorithm. The coefficient of the high threshold multiplied by 0.6 is the low threshold. At this point, the high and low thresholds can realize the dynamic change of the high and low thresholds. In the high and low threshold comparison stage of the Canny edge extraction algorithm, when the gradient value of a pixel is greater than the high threshold, the pixel is determined to be an edge point; when the gradient value of a pixel is less than the low threshold, the pixel is defined as an ordinary pixel; if a pixel is between the high and low thresholds, the gradient value of the surrounding neighborhood pixels of the pixel is determined to determine whether it is an edge point.

[0190] We used images from an actual grain warehouse to conduct experiments. We used the original Canny edge extraction algorithm and the improved edge extraction algorithm to extract the image edges. The comparison of the extraction effects is shown in the figure below. Figure 8 The left image is the image edge extracted by the original Canny edge extraction algorithm, and the right image is the image extracted by the improved edge extraction algorithm. The overall lines are more obvious and neat than the original lines, especially in the digital number part of the door frame. The digital number is an important part of this chapter, and it is necessary to ensure that the number part is as complete as possible. Therefore, the improved edge extraction algorithm can meet the needs well.

[0191] The visual SLAM-based loop detection method provided in the embodiment of the present application obtains an image to be detected; determines a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determines position information corresponding to the feature matrix based on the feature matrix; performs image edge recognition on the image to be detected and determines the object identification in the image to be detected; performs loop determination based on the position information and the object identification and determines the building corresponding to the object identification, and completes loop detection in a grain storage environment by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection.

[0192] Another embodiment of the present application provides a loop closure detection device based on visual SLAM, which is used to execute the loop closure detection method based on visual SLAM provided in the above embodiment.

[0193] Reference Fig. 9 , shows a structural block diagram of an embodiment of a loop detection device based on visual SLAM of the present application, and the device may specifically include the following modules: an acquisition module 901, a first determination module 902, a second determination module 903 and a loop module 904, wherein:

[0194] The acquisition module 901 is used to acquire the image to be detected;

[0195] The first determination module 902 is used to determine a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix based on the feature matrix;

[0196] The second determination module 903 is used to perform image edge recognition on the image to be detected and determine the object identification in the image to be detected;

[0197] The loop module 904 is used to perform loop determination according to the position information and the object identification to obtain a loop determination result, and the loop determination result is used to update the historical data.

[0198] The loop detection device based on visual SLAM provided in the embodiment of the present application obtains the image to be detected; determines the feature matrix corresponding to the image to be detected according to the image to be detected and the pre-trained feature extraction network model, and determines the position information corresponding to the feature matrix according to the feature matrix; performs image edge recognition on the image to be detected to determine the object identification in the image to be detected; performs loop determination according to the position information and the object identification, determines the building corresponding to the object identification, and completes the loop detection in the grain storage environment by obtaining the object and the preset distance in the image to be detected, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection.

[0199] Another embodiment of the present application further supplements the loop detection device based on visual SLAM provided in the above embodiment.

[0200] Optionally, the first determining module is used to:

[0201] Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using the first sample image;

[0202] The principal component analysis method is used to reduce the dimension of the feature matrix and obtain the feature vector after dimension reduction;

[0203] The binarization algorithm is used to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector;

[0204] Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

[0205] Optionally, the first determining module is used to:

[0206] Calculate the covariance matrix of the reduced eigenvector;

[0207] Perform singular value decomposition on the covariance matrix of the eigenvector to obtain a left singular matrix and a right singular matrix;

[0208] The eigenvector after dimension reduction is processed according to the left singular matrix to obtain a processed matrix;

[0209] Calculate the mean of each row of the processed matrix;

[0210] According to the mean of each row, a low-dimensional feature vector is determined.

[0211] Optionally, the second determining module is used to:

[0212] According to the pre-stored template library, the image to be detected is matched to determine the object contour of the image to be detected;

[0213] An edge extraction algorithm is used to extract the edge of the object contour to obtain edge information corresponding to the object contour;

[0214] According to the edge information, the object in the image to be detected is cropped to obtain a cropped image;

[0215] According to a pre-trained digital recognition model and the cropped image, the object identification in the cropped image is determined, wherein the pre-trained digital recognition model is obtained by training a convolutional neural network model using the second sample data.

[0216] Optionally, the second determining module is used to:

[0217] Calculate the direction vector of each pixel in the image to be detected;

[0218] Calculate the dot product between the direction vector of the pixel point in the template in the template library and the direction vector of the pixel point in the image to be detected;

[0219] According to the dot product, determine whether the pixel points in the image to be detected match the pixel points in the template;

[0220] According to the matching results of the pixel points, the edge information corresponding to the object contour is determined.

[0221] Optionally, the second determining module is used to:

[0222] When the pixel gradient value of a pixel point in the image to be detected is greater than a high threshold, the pixel point is determined to be an edge point;

[0223] When the pixel gradient value of a pixel point in the image to be detected is less than a low threshold, the pixel point is determined to be a normal pixel point;

[0224] If the pixel gradient value of a pixel in the image to be detected is between the high threshold and the low threshold, then the pixel is judged to be an edge point based on the pixel gradient values ​​of the neighborhood pixels around the pixel.

[0225] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0226] The loop detection device based on visual SLAM provided in the embodiment of the present application obtains an image to be detected; determines a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determines position information corresponding to the feature matrix based on the feature matrix; performs image edge recognition on the image to be detected and determines the object identification in the image to be detected; performs loop determination based on the position information and the object identification and determines the building corresponding to the object identification, and completes loop detection in a grain storage environment by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection.

[0227] Another embodiment of the present application provides a terminal device for executing the visual SLAM-based loop detection method provided in the above embodiment.

[0228] Fig.10 It is a schematic diagram of the structure of a terminal device of the present application, such as Fig.10 As shown, the terminal device includes: at least one processor 1001 and a memory 1002;

[0229] The memory stores a computer program; and at least one processor executes the computer program stored in the memory to implement the visual SLAM-based loop detection method provided in the above embodiment.

[0230] The terminal device provided in this embodiment obtains an image to be detected; determines a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determines position information corresponding to the feature matrix based on the feature matrix; performs image edge recognition on the image to be detected to determine the object identification in the image to be detected; performs loop determination based on the position information and the object identification to determine the building corresponding to the object identification, and completes loop detection in a grain storage environment by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, thereby effectively solving the problem of loop misjudgment and improving the accuracy and precision of loop detection.

[0231] Another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, the visual SLAM-based loop detection method provided in any of the above embodiments is implemented.

[0232] According to the computer-readable storage medium of this embodiment, by acquiring an image to be detected; determining a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix based on the feature matrix; performing image edge recognition on the image to be detected to determine the object identification in the image to be detected; performing loop determination based on the position information and the object identification to determine the building corresponding to the object identification, by acquiring the object in the image to be detected and a preset distance, and identifying the object identification in the image, loop detection in a grain storage environment is completed together, effectively solving the problem of loop misjudgment, and improving the accuracy and precision of loop detection.

[0233] It should be noted that the above detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present application belongs.

[0234] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0235] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0236] In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0237] For ease of description, spatially relative terms, such as "above", "above", "on the upper surface of", "above", etc., may be used herein to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" may include both "above" and "below". The device may also be positioned in other different ways, such as rotated 90 degrees or in other orientations, and the spatially relative descriptions used herein are interpreted accordingly.

[0238] In the above detailed description, reference is made to the accompanying drawings, which form a part of this document. In the accompanying drawings, similar symbols typically identify similar components unless the context indicates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be used, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

[0239] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A loop detection method based on visual SLAM, characterized in that: The method comprises: Acquire the image to be detected; Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix according to the feature matrix; Performing image edge recognition on the image to be detected to determine the object identification in the image to be detected; A loop determination is performed according to the position information and the object identifier to obtain a loop determination result, and the loop determination result is used to update the historical data.

2. The loop detection method based on visual SLAM according to claim 1, characterized in that: The step of determining a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determining position information corresponding to the feature matrix based on the feature matrix, includes: Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using a first sample image; The principal component analysis method is used to reduce the dimension of the feature matrix to obtain a feature vector after the dimension reduction; Using a binarization algorithm to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector; The Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

3. The loop detection method based on visual SLAM according to claim 2, characterized in that, The binarization algorithm is used to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector, including: Calculating the covariance matrix of the eigenvector after dimensionality reduction; Performing singular value decomposition on the covariance matrix of the eigenvector to obtain a left singular matrix and a right singular matrix; Processing the reduced-dimensional eigenvector according to the left singular matrix to obtain a processed matrix; Calculate the mean of each row of the processed matrix; The low-dimensional feature vector is determined according to the mean of each row.

4. The loop detection method based on visual SLAM according to claim 1, characterized in that: The performing image edge recognition on the image to be detected to determine the object identification in the image to be detected includes: Matching the image to be detected according to a pre-stored template library to determine the object contour of the image to be detected; Using an edge extraction algorithm to extract the edge of the object contour to obtain edge information corresponding to the object contour; According to the edge information, the object in the image to be detected is cropped to obtain a cropped image; According to a pre-trained digital recognition model and the cropped image, the object identification in the cropped image is determined, wherein the pre-trained digital recognition model is obtained by training a convolutional neural network model using the second sample data.

5. The loop detection method based on visual SLAM according to claim 4, characterized in that: The step of extracting the edge of the object contour using an edge extraction algorithm to obtain edge information corresponding to the object contour includes: Calculate the direction vector of each pixel in the image to be detected; Calculating the dot product of the direction vector of the pixel point in the template in the template library and the direction vector of the pixel point in the image to be detected; According to the dot product, determining whether a pixel point in the image to be detected matches a pixel point in the template; According to the matching result of the pixel points, edge information corresponding to the contour of the object is determined.

6. The loop detection method based on visual SLAM according to claim 5, characterized in that: The method further comprises: When the pixel gradient value of a pixel point in the image to be detected is greater than a high threshold, the pixel point is determined to be an edge point; When the pixel gradient value of a pixel point in the image to be detected is less than a low threshold, the pixel point is determined to be a normal pixel point; If the pixel gradient value of a pixel point in the image to be detected is between the upper threshold and the lower threshold, then it is determined whether the pixel point is an edge point based on the pixel gradient values ​​of the neighborhood surrounding the pixel point.

7. A loop detection device based on visual SLAM, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be detected; A first determination module is used to determine a feature matrix corresponding to the image to be detected based on the image to be detected and a pre-trained feature extraction network model, and determine position information corresponding to the feature matrix based on the feature matrix; A second determination module is used to perform image edge recognition on the image to be detected and determine the object identification in the image to be detected; The loop module is used to perform loop determination according to the position information and the object identifier to obtain a loop determination result, and the loop determination result is used to update the historical data.

8. The loop detection device based on visual SLAM according to claim 7, characterized in that: The first determining module is used for: Determine a feature matrix corresponding to the image to be detected according to the image to be detected and a pre-trained feature extraction network model; wherein the pre-trained feature extraction network model is obtained by training a GhostNet network model using a first sample image; The principal component analysis method is used to reduce the dimension of the feature matrix to obtain a feature vector after the dimension reduction; Using a binarization algorithm to process the reduced-dimensional feature vector to obtain a low-dimensional feature vector; The Hamming distance is used to judge the low-dimensional feature vector and the position information corresponding to the feature matrix.

9. A terminal device, characterized in that: include: at least one processor and memory; The memory stores a computer program; The at least one processor executes the computer program stored in the memory to implement the loop detection method based on visual SLAM according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the loop detection method based on visual SLAM described in any one of claims 1 to 6 is implemented.