Visual location identification model construction method and system based on tetrad loss network

Abstract

The invention provides a method and a system for constructing a visual location recognition model based on a tetrad loss network, relates to the technical field of robot and computer vision, and aims to solve the problems that the traditional visual recognition technology cannot keep robustness under the scenes of environmental condition factor change, visual angle change and the like, and the model recognition accuracy is poor. Obtaining a global feature descriptor through a quaternary pyramid encoder, and inputting the global feature descriptor into a Hash encoder to generate a global Hash code; training the quaternary loss network based on the global hash code; a Bayesian quaternary loss network is adopted to carry out probability estimation, and a constructed maximum likelihood function is used to extract candidate sites. Through fusion of a multi-channel attention mechanism and mask features, the multi-scale feature expression ability of a site recognition model is enhanced, a quaternary loss function is introduced to improve the generalization performance and matching accuracy of the model, and the problem that a traditional visual recognition technology is poor in robustness and accuracy under the scenes of environmental condition factor change, view angle change and the like is solved.

Description

Technical Field

[0001] The present invention belongs to the field of robot and computer vision technology, and in particular relates to a method and system for constructing a visual place recognition model based on a quadruple loss network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Visual place recognition is a challenging task in the field of computer vision. It extracts the features of the target and then matches the feature points to achieve location recognition and loop detection. As an important research direction in the field of computer vision, this task is critical in many applications. In the field of robot navigation, visual place recognition can help robots capture images of the surrounding environment through cameras and match them with pre-built maps to achieve autonomous movement and loop detection. In the field of autonomous driving, autonomous driving systems use visual place recognition to identify lane markings, traffic signs, and road signs for lane keeping and path planning.

[0004] In recent years, visual place recognition has faced some key challenges that have affected its performance and reliability in practical applications. Visual place recognition systems have difficulty maintaining robustness in different lighting conditions, weather changes (such as rain, snow, fog), and seasonal changes (such as spring, summer, autumn, and winter); environmental changes can significantly affect the characteristics of the image, thereby affecting the accuracy of recognition; the same place photographed from different distances, angles, or perspectives may produce significant visual differences, making it difficult for traditional visual feature matching methods to effectively identify and compare.

[0005] Through the above analysis, it can be seen that the existing visual recognition technology has certain shortcomings:

[0006] (1) Environmental changes can significantly affect image features, and existing video location recognition technologies cannot maintain robustness under different lighting conditions, weather changes (such as rain, snow, and fog), and seasonal changes (such as spring, summer, autumn, and winter), resulting in poor recognition accuracy.

[0007] (2) Shooting the same location from different distances, angles, or perspectives may produce significant visual differences, and traditional visual feature matching methods are difficult to effectively identify and compare in scenarios such as changes in perspective. Summary of the invention

[0008] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for constructing a visual place recognition model based on a quadruple loss network. Through the multi-channel attention layer (MCAL) and mask feature fusion, the multi-scale feature expression capability of the place recognition model is enhanced, and the quadruple loss function is introduced to improve the generalization performance and matching accuracy of the model, thus overcoming the problem of poor robustness and accuracy of traditional visual recognition technology in scenarios such as changes in environmental conditions and viewing angles.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] In a first aspect, a method for constructing a visual place recognition model based on a quadruple loss network is disclosed, comprising:

[0011] Acquire a quadruple image, wherein the quadruple image includes continuous anchor point images, a positive sample image, and two negative sample images;

[0012] The quaternary image is input into the quaternary pyramid encoder, and the attention features and mask features of each pyramid are extracted through downsampling and upsampling of the pyramid encoder. The attention features and mask features of each pyramid encoder are fused to obtain the reconstructed features of each pyramid. The global feature descriptor is obtained by fusing the reconstructed features of each pyramid encoder.

[0013] Input the global feature descriptor into the hash encoder, perform hash feature encoding, and generate a global hash code;

[0014] Input the global hash code into the quaternion loss network for training;

[0015] Bayesian method is used to estimate the probability of the trained four-tuple loss network to obtain the four-tuple constraint. According to the four-tuple constraint, the likelihood function is constructed, and the candidate locations are extracted according to the maximum value of the likelihood function.

[0016] As a further technical solution, the extraction process of each pyramid attention feature is:

[0017] Extract multi-scale features using downsampling of the pyramid encoder;

[0018] Input the multi-scale features into the multi-head attention layer of the pyramid encoder for encoding, and obtain the multi-head attention layer output;

[0019] The output of the multi-head attention layer is input into the multi-layer perception layer for dimensionality reduction to obtain each multi-head attention feature;

[0020] Each multi-head attention feature is concatenated through linear transformation to obtain the attention feature of each pyramid.

[0021] As a further technical solution, the extraction process of each pyramid mask feature is:

[0022] Upsample the multi-scale features and input them into the pyramid multi-head attention layer for encoding to obtain the multi-head attention layer output;

[0023] The output of the multi-head attention layer is input into the multi-layer perception layer for dimensionality reduction to obtain the mask features of each multi-head attention layer;

[0024] The mask features of each multi-head attention layer are concatenated through linear transformation to obtain the mask features of each pyramid.

[0025] As a further technical solution, after the multi-scale features are input into the pyramid encoder, the pyramid encoder is pre-trained. Specifically, the pyramid encoder consists of a backbone network, a neck module and a head module. The training data is input into the backbone network to generate a multi-scale feature map. The multi-scale features are fused through the neck module, and the fused features output by the neck module are linearly transformed to output feature representation.

[0026] As a further technical solution, the trained pyramid encoder is fine-tuned. Specifically, the neck module of the pyramid encoder is fine-tuned by constructing a minimum fine-tuning function.

[0027] As a further technical solution, the global feature descriptor is calculated as follows:

[0028]

[0029] in, represents the global feature descriptor of the sth stage, and They represent the attention feature descriptor and the mask feature descriptor respectively, z msa and z mlp Respectively represent the multi-head attention encoding output and the multi-layer perception layer output; U S A set of labels representing each feature map, represents the Kth feature map of the Sth feature map s Mark, K s is the number of tags in the Sth feature map; V S represents the fine-tuning output in the s-th feature map.

[0030] In a second aspect, a system for constructing a visual place recognition model based on a quadruple loss network is disclosed, comprising:

[0031] A data acquisition module, used to acquire a four-tuple image, wherein the four-tuple image includes a continuous anchor point image, a positive sample image and two negative sample images;

[0032] A feature extraction module is used to input the quaternary image into the quaternary pyramid encoder, extract the attention feature and mask feature of each pyramid through downsampling and upsampling of the pyramid encoder, perform feature fusion on the attention feature and mask feature of each pyramid encoder to obtain the reconstructed feature of each pyramid, and obtain the global feature descriptor by fusing the reconstructed features of each pyramid encoder;

[0033] A hash coding module, used for inputting the global feature descriptor into a hash encoder, performing hash feature coding, and generating a global hash code;

[0034] A quaternary loss network module, which is used to input the global hash code into the quaternary loss network for training;

[0035] The Bayesian optimization and location extraction module is used to use Bayesian to perform probability estimation on the trained four-tuple loss network to obtain four-tuple constraints, construct a likelihood function based on the four-tuple constraints, and extract candidate locations based on the maximum value of the likelihood function.

[0036] The third aspect of the present invention is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0037] A fourth aspect of the present invention is to provide a computer-readable storage medium having a computer program stored thereon, which executes the steps of the above method when executed by a processor.

[0038] A fifth aspect of the present invention is to provide a computer program product, comprising a computer program, which performs the above-described method when executed by a processor.

[0039] One or more of the above technical solutions have the following beneficial effects:

[0040] In this embodiment, the multi-scale feature expression capability of the place recognition model is enhanced through a multi-channel attention mechanism and mask feature fusion, and the generalization performance and matching accuracy of the model are improved by introducing a four-element loss function. This solves the problem that visual place recognition cannot maintain robustness and accuracy in scenarios such as changing environmental conditions and viewing angles, thereby improving the generalization performance and matching accuracy of the model.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 Schematic diagram of the framework of the visual place recognition model based on the quadruple loss network in the first embodiment;

[0044] Figure 2 This is a flow chart of a method for constructing a visual place recognition model based on a quadruple loss network in the first embodiment;

[0045] Figure 3 A diagram showing the comparison results between the visual place recognition model construction method based on the quadruple loss network in the first embodiment and other recognition methods;

[0046] Figure 4 This is an example of successful matching of daytime and nighttime scenes in the first embodiment;

[0047] Figure 5 This is the storage format of the location image data set in the first embodiment. DETAILED DESCRIPTION

[0048] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0049] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0050] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] The overall idea proposed by the present invention is as follows: first, a global feature descriptor is obtained through a quaternary pyramid encoder, and the global feature descriptor is input into a hash encoder to generate a global hash code; the global hash code is input into a quaternary loss network for training; Bayesian is used to perform probability estimation on the trained quaternary loss network to obtain a quaternary constraint, and a likelihood function is constructed according to the quaternary constraint, and candidate locations are extracted according to the maximum value of the likelihood function.

[0052] Embodiment 1

[0053] This embodiment discloses a method for constructing a visual place recognition model based on a quadruple loss network.

[0054] In order to more clearly illustrate this embodiment, the process of constructing and implementing the visual place recognition model based on the quadruple loss network can be specifically described as follows:

[0055] A method for constructing a visual place recognition model based on a quadruple loss network, comprising:

[0056] S1. Acquire a quadruple image, where the quadruple image includes a continuous anchor point image, a positive sample image, and two negative sample images;

[0057] S2, input the quaternary image into the quaternary pyramid encoder, extract the attention features and mask features of each pyramid through downsampling and upsampling of the pyramid encoder, perform feature fusion on the attention features and mask features of each pyramid encoder to obtain the reconstructed features of each pyramid, and obtain the global feature descriptor by fusing the reconstructed features of each pyramid encoder;

[0058] S3, inputting the global feature descriptor into the hash encoder, performing hash feature encoding, and generating a global hash code;

[0059] S4, input the global hash code into the quaternion loss network for training;

[0060] S5. Use Bayesian to estimate the probability of the trained four-tuple loss network to obtain the four-tuple constraint. According to the four-tuple constraint, construct the likelihood function, and extract the candidate locations according to the maximum value of the likelihood function.

[0061] like Figure 1 As shown, in step S1, a quadruple image is obtained, and the quadruple image includes continuous anchor point images, a positive sample image and two negative sample images.

[0062] In this embodiment, a location image dataset is selected, and the location image dataset is divided into four parts, such as Figure 5 The storage format of the place image dataset is shown, where placeA is used to store anchor point sample images, placeB is used to store positive sample images, placeC is a negative sample image, and placeD stores negative sample images.

[0063] Take one image from each of the four parts of the place image dataset in order to form a four-tuple. For example, when taking it for the first time, take out the first anchor sample image in placeA, the first positive sample image in placeB, the first negative sample image in placeC, and the first negative sample image in placeD to form a four-tuple image {x a ,x p ,x n1 ,x n2}, take them out one by one to get a four-tuple data set.

[0064] The four-tuple dataset is divided into two parts, where 60% of the data samples Used as a quaternary pyramid encoder pre-training, where N represents the number of unlabeled samples; the remaining 40% data samples is used as a quaternary pyramid encoder fine-tuning, where M represents the number of samples, express When pre-training and fine-tuning different pyramid encoders, and They all represent the input images of the corresponding pyramid encoders.

[0065] like Figure 1 As shown, after step S1, after the quadruple image is input into the pyramid encoder, the pyramid encoder is pre-trained, specifically: the pyramid encoder consists of a backbone network, a neck module and a head module, the training data is input into the backbone network, a multi-scale feature map is generated, the multi-scale features are fused through the neck module, and the fused features output by the neck module are linearly transformed to output feature representation.

[0066] In this embodiment, the quaternary pyramid encoder contains four identical pyramid encoders, each of which is composed of a backbone network (backbone), a neck module (neck) and a head module (head), respectively using f(θ), g(φ) and Indicates that, θ, φ, Represents the learnable parameters in the model.

[0067] In this embodiment, The four-tuple image in the dataset is input to the four-tuple pyramid encoder, and the backbone network f(θ) in each pyramid encoder is x H×W As input, where W and H represent the size of the input image, the neck g(φ) is responsible for pyramid feature aggregation, and the head Used for final feature output.

[0068] Downsampling is performed first and then upsampling is performed to obtain multi-scale features.

[0069] First, the model backbone network processes the input image x of size H×W to generate a multi-scale feature map, the formula is:

[0070]

[0071] Among them, U s and V S Respectively represent the feature maps and mask maps of the corresponding stages in the down-sampling and up-sampling processes, and each feature map consists of a set of markers Composition, K sis the number of tokens in the feature map, and the backbone network f(θ) of the pyramid encoder is based on x H×W As input, where W and H represent the size of the input image and g(·) represents the mask generation function.

[0072] Then, the up- and down-sampled features are fused through the neck module to obtain the reconstructed feature map of each pyramid.

[0073] Finally, the reconstructed feature map is passed through the head module for final feature output. The whole model formula is:

[0074]

[0075] where h(·), g(·), and f(·) represent the functions of the “head”, “neck”, and “skeleton” networks of the model, respectively, and x represents the input image representation. represents the learnable model parameters of the “neck” network, and φ represents the learnable model parameters of the “head” network.

[0076] The pyramid encoder pre-training process can be described as:

[0077]

[0078] Among them, φ pre represents the pre-trained parameters in the “head” network, represents the pre-trained parameters in the “neck” network, φ pre and There is no weight ratio between them.

[0079] Each pyramid encoder is pre-trained according to formula (4).

[0080] Fine-tune the trained pyramid encoder model.

[0081] Specifically, by constructing a minimum fine-tuning function, the neck module of each pyramid encoder is fine-tuned, and the parameters of other modules are frozen.

[0082] Will Input to each pyramid encoder for fine-tuning, where M represents the number of samples, express .

[0083]

[0084] where φ pre ,φ fin , There is no weight ratio between them.

[0085] Each pyramid encoder is fine-tuned according to formula (5) to optimize the performance of the pyramid encoder and extract more effective features.

[0086] In order to reduce the data transmission error between pre-training and fine-tuning, pre-training and fine-tuning are combined into a unified framework so that the pre-trained parameters can be easily used in the fine-tuning stage. The pyramid encoder contains S modules, each of which consists of several transformer blocks. When the image I is input, the backbone network module will encode the input image and gradually downsample it to generate S+1 feature maps:

[0087]

[0088] Among them, U 0 Represents the first embedded patch of the input image. Each feature map consists of a set of markers, such as:

[0089]

[0090] Among them, K s is the number of tokens in the Sth feature map, represents the Kth feature map of the Sth feature map s mark.

[0091] In the quaternion pyramid encoder, the pre-training stage and the fine-tuning stage share the same architecture and parameters, and both aggregate features from lower-level stages. Therefore, the fine-tuning stage (neck) can be expressed as:

[0092]

[0093] The neck gradually downsamples the input feature map (from the upsampled output) so that V s+1 Comply with V s To reduce the transfer gap, the parameter set {φ} from the pre-training stage is reused in the fine-tuning stage, while the recognition head remains a separate entity.

[0094] like Figure 1 , Figure 2 As shown, in step S2, the four-tuple image is input into the four-tuple pyramid encoder, the attention features and mask features of each pyramid are extracted through downsampling and upsampling of the pyramid encoder, the attention features and mask features of each pyramid encoder are fused to obtain the reconstructed features of each pyramid, and the global feature descriptor is obtained by fusing the reconstructed features of each pyramid encoder.

[0095] S201, obtaining attention features through downsampling, obtaining mask features through upsampling, and fusing the attention features and mask features of each pyramid encoder to obtain the reconstructed features of each pyramid.

[0096] The feature representation effect is enhanced by fusing attention features and mask features for cross-domain reconstruction. A reconstruction head is added at each scale stage to make the multi-level loss L M Get joint optimization:

[0097]

[0098] Among them, L I =||xh pre,0 (V)|| represents the loss of image reconstruction, represents the loss of feature reconstruction, λ represents the exponent determined by the input dataset, and is set to λ = 0.3.

[0099] The pyramid encoder is used to downsample and extract multi-scale features. The multi-scale features are input into the multi-head attention layer of the pyramid encoder for encoding to obtain the multi-head attention layer output. The multi-head attention layer output is input into the multi-layer perception layer for dimensionality reduction to obtain each multi-head attention feature. Each multi-head attention feature is concatenated through linear changes to obtain the attention feature of each pyramid.

[0100] The multi-scale features are upsampled and input into the pyramid multi-head attention layer for encoding to obtain the multi-head attention layer output, and the multi-head attention layer output is input into the multi-layer perception layer for dimensionality reduction to obtain the mask features of each multi-head attention layer. The mask features of each multi-head attention layer are concatenated through linear changes to obtain the mask features of each pyramid.

[0101] like Figure 1 As shown, in this embodiment, specifically, multi-scale features are weighted, and the outputs of multiple transformer stages are encoded to obtain multi-head attention encoding outputs and multi-layer perception layer outputs.

[0102] The pyramid encoder uses the vision transformer (ViT) as the basic skeleton and weights the multi-scale features to encode the outputs of multiple transformer stages. Each module consists of L self-attention layers, which contain a multi-head attention layer and a multi-layer perception layer (MLP). Each self-attention layer is followed by layer normalization (LN). The encoding process of each transformer module is expressed as follows:

[0103] z msa =f msa (f ln (zl-1 ))+z l-1

[0104] z mlp =f mlp (f ln (z l ))+z l (10)

[0105] Among them, z msa and z mlp They represent the multi-head attention encoding output and the multi-layer perception layer output, respectively. msa and f mlp They represent the multi-head self-attention layer and the multi-layer perceptron layer (MLP), respectively. l-1 Represents the input of the l-1th layer. l Represents the input of the lth layer, that is, the feature representation of the current layer.

[0106] S202: Obtain a global feature descriptor by fusing the reconstructed features of each pyramid encoder.

[0107] The reconstructed features of all pyramids are fused to obtain the global feature descriptor, the formula is:

[0108]

[0109] in, represents the global feature descriptor of the sth stage, and They represent the attention feature descriptor and the mask feature descriptor respectively, z msa and z ml p represents the multi-head attention encoding output and the multi-layer perception layer output; U S A set of labels representing each feature map, represents the Kth feature map of the Sth feature map s Mark, K s is the number of tags in the Sth feature map; V S represents the fine-tuning ("neck" network) output in the sth feature map.

[0110] like Figure 1 , Figure 2 As shown, in step S3, the global feature descriptor is input into the hash encoder, hash feature encoding is performed, and a global hash code is generated.

[0111] To reduce storage space and retrieval time requirements, the feature vector output by the pyramid encoder module is projected into a binary code through deep hash coding to learn a compact hash code. Given that each image in the matching stage has a b-bit hash code, a b-bit global hash vector of the feature vector with a labeled embedding size d can be calculated as follows:

[0112] f′(x)=f sigmoid (z g ω+B)(12)

[0113] Among them, f sigmoid (·) represents the activation function, ω represents the weight matrix, B represents the bias parameter set, B={b1,b2,...,b i}.

[0114] After the above steps, a binary hash code is generated, thereby reducing storage space and speeding up retrieval.

[0115] like Figure 1 , Figure 2 As shown, in step S4, the global hash code is input into the quaternary loss network for training.

[0116] Since the triple loss network cannot handle complex category relationships, a quadruple loss network is constructed based on the triple loss network, and the global hash code is input into the quadruple loss network for training to obtain the optimal quadruple loss network.

[0117] The input of the quadruple loss network contains four basic primitives {x a ,x p ,x n1 ,x n2}, which includes two negative elements x n1 and x n2 , the quaternary loss network formula is:

[0118]

[0119] Among them, α and β are the residual values ​​of the two terms, x a and x p Represents anchor primitives and positive primitives, x n1 and x n2 represents two negative primitives, f′(x a ) represents the global hash code of the input anchor primitive, f′(x p ) represents the global hash code of the input positive primitive, f′(x n1 ) and f′(x n2 ) represents the negative primitive global hash code of the input.

[0120] After the above steps, a negative constraint is added on the basis of the triplet network to enhance the ability to distinguish small differences between positions (images). The model accurately identifies positions in complex environments, can distinguish more complex category relationships, and improves the generalization ability of the model.

[0121] like Figure 1 , Figure 2 As shown, in step S5, Bayesian method is used to perform probability estimation on the trained four-tuple loss network to obtain four-tuple constraints, and a likelihood function is constructed based on the four-tuple constraints. The candidate locations are extracted based on the maximum value of the likelihood function.

[0122] S51. Use Bayesian to perform probability estimation on the trained four-tuple loss network to obtain the four-tuple constraint.

[0123] In order to improve the loss measurement effect to a greater extent, Bayesian estimation is added to the quaternary loss model, proposing to embed the image as a distribution estimate rather than a point estimate. Given these random embeddings, the proposed Bayesian quaternary loss model aims to solve the anchor x a Closer to positive x p The probability P1 and negative x n1 Closer to another x n2 The probability P2 is equivalent to the quaternion constraint in probability, and the formula is:

[0124]

[0125] Where P1 represents the anchor x a Closer to positive x p The probability of negative Closer to another probability.

[0126] S52. Construct a likelihood function according to the four-tuple constraint.

[0127] Using the likelihood function, we define the likelihood of all the four-element images in the data set. The formula is:

[0128]

[0129] Among them, X1, X2, X3, X4 are images in the dataset, and I is the label of the four-tuple.

[0130] The likelihood function is defined as a multi-branch distribution, which reduces the probability likelihood to only consider quaternions with a pair of images:

[0131]

[0132] Through Bayesian learning, the four-element loss model can introduce probability distribution on the distance estimation between anchor points, positive samples, negative samples and far negative samples, effectively eliminating the uncertainty of probability estimation.

[0133] S52. Extract candidate locations based on the maximum value of the likelihood function.

[0134] In the location recognition task, assuming there is a set of candidate locations, the system can calculate the likelihood value of each candidate location based on the observed data (image). Then, the candidate location with the highest log-likelihood value is selected as the most likely location under the observed data to achieve the location recognition task.

[0135] The visual place recognition model based on the quadruple loss network is applied to public place recognition datasets, such as KITTI, Nordland, VPRICE and EuRoC, for performance evaluation. The results are shown in Table 1.

[0136] Comparing the recognition method in this embodiment with the most advanced benchmarks, it is obvious that the recognition method in this embodiment outperforms all aggregate feature search and re-ranking methods. AQPT-agg and AQPT-g represent the aggregate feature method (global feature and mask feature aggregation) and the global feature method, respectively. In addition, it can be seen from the comparison in Table 1 that this method outperforms other methods in both R@1 and R@5 evaluation indicators.

[0137] Table 1 Performance comparison of this method with other advanced methods on various datasets

[0138]

[0139] The method in the embodiment is applied to the recognition and matching of daytime and nighttime scenes. The model in this embodiment successfully matches the position pairs and extracts the first 800 significant features (highlights in the figure) from the corresponding images. The red rectangular boxes represent the scenes visited by the robot during the day, and the green rectangular boxes represent the scenes visited by the robot during the day and at night. Figure 3 shown.

[0140] Embodiment 2

[0141] The purpose of this embodiment is to provide a visual place recognition model construction system based on a quadruple loss network, including:

[0142] A data acquisition module, used to acquire a four-tuple image, wherein the four-tuple image includes a continuous anchor point image, a positive sample image and two negative sample images;

[0143] A feature extraction module is used to input the quaternary image into the quaternary pyramid encoder, extract the attention feature and mask feature of each pyramid through downsampling and upsampling of the pyramid encoder, perform feature fusion on the attention feature and mask feature of each pyramid encoder to obtain the reconstructed feature of each pyramid, and obtain the global feature descriptor by fusing the reconstructed features of each pyramid encoder;

[0144] A hash coding module, used for inputting the global feature descriptor into a hash encoder, performing hash feature coding, and generating a global hash code;

[0145] A quaternary loss network module, which is used to input the global hash code into the quaternary loss network for training;

[0146] The Bayesian optimization and location extraction module is used to use Bayesian to perform probability estimation on the trained four-tuple loss network to obtain four-tuple constraints, construct a likelihood function based on the four-tuple constraints, and extract candidate locations based on the maximum value of the likelihood function.

[0147] Based on providing a visual place recognition model construction system based on a quadruple loss network, the method steps in embodiment 1 are implemented.

[0148] Embodiment 3

[0149] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0150] Embodiment 4

[0151] The purpose of this embodiment is to provide a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above method are performed.

[0152] Embodiment 5

[0153] The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments.

[0154] The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0155] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0156] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing a visual place recognition model based on a quadruple loss network, characterized in that: include: Acquire a quadruple image, wherein the quadruple image includes continuous anchor point images, a positive sample image, and two negative sample images; The quaternary image is input into the quaternary pyramid encoder, and the attention features and mask features of each pyramid are extracted through downsampling and upsampling of the pyramid encoder. The attention features and mask features of each pyramid encoder are fused to obtain the reconstructed features of each pyramid. The global feature descriptor is obtained by fusing the reconstructed features of each pyramid encoder. Input the global feature descriptor into the hash encoder, perform hash feature encoding, and generate a global hash code; Input the global hash code into the quaternion loss network for training; Bayesian method is used to estimate the probability of the trained four-tuple loss network to obtain the four-tuple constraint. According to the four-tuple constraint, the likelihood function is constructed, and the candidate locations are extracted according to the maximum value of the likelihood function.

2. The method for constructing a visual place recognition model based on a quadruple loss network according to claim 1, characterized in that: The extraction process of each pyramid attention feature is: Extract multi-scale features using downsampling of the pyramid encoder; Input the multi-scale features into the multi-head attention layer of the pyramid encoder for encoding, and obtain the multi-head attention layer output; The output of the multi-head attention layer is input into the multi-layer perception layer for dimensionality reduction to obtain each multi-head attention feature; Each multi-head attention feature is concatenated through linear transformation to obtain the attention feature of each pyramid.

3. The method for constructing a visual place recognition model based on a quadruple loss network according to claim 1, characterized in that: The extraction process of each pyramid mask feature is: Upsample the multi-scale features and input them into the pyramid multi-head attention layer for encoding to obtain the multi-head attention layer output; The output of the multi-head attention layer is input into the multi-layer perception layer for dimensionality reduction to obtain the mask features of each multi-head attention layer; The mask features of each multi-head attention layer are concatenated through linear transformation to obtain the mask features of each pyramid.

4. The method for constructing a visual place recognition model based on a quadruple loss network according to claim 1, characterized in that: After the quadruple image is input into the pyramid encoder, the pyramid encoder is pre-trained. Specifically, the pyramid encoder consists of a backbone network, a neck module and a head module. The training data is input into the backbone network to generate a multi-scale feature map. The multi-scale features are fused through the neck module, and the fused features output by the neck module are linearly transformed to output feature representation.

5. The method for constructing a visual place recognition model based on a quadruple loss network according to claim 4, characterized in that: The trained pyramid encoder is fine-tuned. Specifically, the neck module of the pyramid encoder is fine-tuned by constructing a minimum fine-tuning function.

6. The method for constructing a visual place recognition model based on a quadruple loss network according to claim 1, characterized in that: The global feature descriptor calculation formula is: in, represents the global feature descriptor of the sth stage, and They represent the attention feature descriptor and the mask feature descriptor respectively, z msa and z mlp Respectively represent the multi-head attention encoding output and the multi-layer perception layer output; U S A set of labels representing each feature map, represents the Kth feature map of the Sth feature map s Mark, K s is the number of tags in the Sth feature map; V S represents the fine-tuning output in the s-th feature map.

7. A visual place recognition model construction system based on a quadruple loss network, characterized in that: include: A data acquisition module, used to acquire a four-tuple image, wherein the four-tuple image includes a continuous anchor point image, a positive sample image and two negative sample images; A feature extraction module is used to input the quaternary image into the quaternary pyramid encoder, extract the attention feature and mask feature of each pyramid through downsampling and upsampling of the pyramid encoder, perform feature fusion on the attention feature and mask feature of each pyramid encoder to obtain the reconstructed feature of each pyramid, and obtain the global feature descriptor by fusing the reconstructed features of each pyramid encoder; A hash coding module, used for inputting the global feature descriptor into a hash encoder, performing hash feature coding, and generating a global hash code; A quaternary loss network module, which is used to input the global hash code into the quaternary loss network for training; The Bayesian optimization and location extraction module is used to use Bayesian to perform probability estimation on the trained four-tuple loss network to obtain four-tuple constraints, construct a likelihood function based on the four-tuple constraints, and extract candidate locations based on the maximum value of the likelihood function.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images