A visual position recognition method, electronic device, and medium

Abstract

The application discloses a visual position recognition method, an electronic device and a medium, and comprises the following steps: acquiring an input image; extracting a feature vector of the input image by using a convolutional neural network; training a principal component analysis conversion model based on unsupervised learning; reconstructing the feature vector of the input image by using the principal component analysis conversion model to generate an image description vector; acquiring an image description vector of an existing image from a database, and calculating the similarity between the image description vector of the input image and the image description vector of the existing image; when the maximum similarity is greater than or equal to a similarity threshold, regarding the existing image corresponding to the maximum similarity as a similar image of the input image to obtain a visual position recognition result.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to a visual position recognition method, electronic device, and medium. Background Technology

[0002] Visual location recognition is a task that determines whether images belong to the same location. Traditional visual location recognition algorithms, with the bag-of-words model being a prominent example, mainly utilize manually designed features extracted from images to determine image similarity. These algorithms perform well when scene features and manually designed features are similar, but as environmental complexity increases, the insufficient expressive power of manually designed features will lead to the failure of the overall algorithm.

[0003] Current common deep learning-based visual position recognition algorithms employ two approaches: the first is transfer learning based on a general convolutional neural network model; the second is designing a network model specifically for visual position recognition and performing supervised training from scratch. Both approaches significantly outperform traditional algorithms in complex real-world scenarios, but the need for labeled datasets in the supervised learning process greatly increases the cost of related research.

[0004] To explore low-cost algorithmic approaches, some researchers have introduced unsupervised learning into the field of visual position recognition, such as denoising convolutional autoencoder structures, successfully addressing the dependence on labeled samples. However, unsupervised learning approaches struggle to match the detection accuracy of supervised learning approaches, especially in highly dynamic real-world scenarios involving variations in viewpoint, objects, lighting, and season. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a visual position recognition method, an electronic device, and a medium.

[0006] According to a first aspect of the present invention, a visual position recognition method is provided, the method comprising:

[0007] Obtain the input image;

[0008] Use a convolutional neural network to extract feature vectors from the input image;

[0009] Based on unsupervised learning, a principal component analysis transformation model is trained, and the feature vector of the input image is reconstructed using the principal component analysis transformation model to generate an image description vector.

[0010] The image description vectors of existing images are obtained from the database. The similarity between the image description vector of the input image and the image description vectors of existing images is calculated. When the maximum similarity is greater than or equal to the similarity threshold, the existing image corresponding to the maximum similarity is used as the similar image of the input image to obtain the visual location recognition result.

[0011] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described visual position recognition method.

[0012] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described visual position recognition method.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention provides a visual position recognition method that integrates unsupervised learning and deep learning. It maps the output vector of a convolutional neural network model to a principal component space with clearer vector backbone directions, improving the cosine similarity between similar image vectors. Furthermore, this method does not require annotation of training data or retraining of the model. This invention improves detection accuracy while maintaining low implementation cost. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart of a visual position recognition method provided in an embodiment of the present invention;

[0017] Figure 2 This is a structural diagram of the AlexC3 convolutional neural network model;

[0018] Figure 3 The results of information compression characteristics of principal component vector spaces with different target dimensions are shown in the figure.

[0019] Figure 4 The graph shows the impact of principal component transformation on the accuracy and performance of the algorithm under different target dimensions.

[0020] Figure 5 The graph shows the impact of principal component transformation on the algorithm's running efficiency for different target dimensions.

[0021] Figure 6 The figure shows the results of an experiment comparing the accuracy performance of the method of this invention with that of various comparison algorithms on multiple visual location recognition datasets.

[0022] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0025] like Figure 1 As shown, this embodiment of the invention provides a visual location recognition method, which includes the following steps:

[0026] Step S1: Obtain the input image and preprocess it.

[0027] Specifically, step S1 includes the following sub-steps:

[0028] S101: Enhance the contrast of the current input image. First, convert the image from the RGB color space to the YUV color space, and perform histogram equalization on the Y channel separately to make the brightness values ​​of the image approximately uniformly distributed in the range of 0 to 255, so as to increase the distinguishability between various features; then convert the image back to the RGB color space and normalize it.

[0029] S102: The current input image size is uniformly transformed to 224*224*3 to ensure that it is consistent with the size of the subsequent AlexC3 model pre-training data, reducing the possibility of neural network failure. The size transformation is based on the bicubic interpolation method, and the interpolated pixel value p(x,y) at the image point (x,y) can be calculated by the following formula:

[0030] p(x,y)=W x ** y

[0031] In the formula, Q represents a 4x4 matrix consisting of the pixel values ​​of 16 points near the corresponding position of point (x,y) in the original image; the weight coefficient matrix W x The calculation formula is as follows:

[0032]

[0033] Step S2: Use a convolutional neural network to extract the feature vector of the input image.

[0034] In this example, the convolutional neural network used is the AlexC3 model.

[0035] Among them, such as Figure 2 As shown, the construction of the convolutional neural network AlexC3 model includes the following steps:

[0036] S201: First, based on the structure and pre-training parameters of the general convolutional neural network model AlexNet, we make full use of the good feature abstraction capabilities obtained by the general convolutional neural network model under large-scale supervised training.

[0037] S202: To improve the generalization performance of the model, this invention uses the first three convolutional layers of the original model as the main structure of the AlexC3 model, and combines them with a Sigmoid layer to process the output, replacing max pooling and ReLU activation operations. The addition of the Sigmoid layer successfully avoids the problem of excessively large element values ​​in the image feature vector and ensures that the data distribution of the original features remains unchanged. The specific calculation formula for the Sigmoid layer is as follows:

[0038]

[0039] In the formula, This represents the value of the i-th element of the feature vector output by the third convolutional layer of the AlexC3 model. This represents the value of the i-th element of the feature vector output by the Sigmoid layer.

[0040] S203: To ensure the high operating efficiency of the network model, the AlexC3 model proposed in this invention truncates all fully connected layers in the original model, thereby improving the inference efficiency of the model while shortening the time consumed in the subsequent unsupervised learning process.

[0041] S204: After determining the basic structure of the AlexC3 model, this invention changes the number of convolutional kernels in the first and second convolutional layers from 96 and 256 to 64 and 192, respectively, to ensure that the model structure is adapted to single-GPU operation, while the number of convolutional kernels in the third convolutional layer remains unchanged.

[0042] S205: After the above steps, the AlexC3 model is obtained. Its specific structure includes a first convolutional layer, a first ReLU layer, a first max-pooling layer, a second convolutional layer, a second ReLU layer, a second max-pooling layer, a third convolutional layer, and a Sigmoid layer connected in sequence. The dimensions of the first, second, and third convolutional layers are 11*11*3*64, 5*5*64*192, and 3*3*192*384, respectively. The pooling kernel size of the max-pooling layers is 3. The calculation formula for the ReLU layer is as follows:

[0043] output = max(0, input)

[0044] In the formula, max() represents the maximum value function, input represents the input data of the ReLU layer, and output represents the output data of the ReLU layer. That is, the ReLU layer will output 0 for all input data less than or equal to 0, and keep the input data greater than 0 unchanged.

[0045] S3: Based on unsupervised learning, a principal component analysis transformation model is trained, and the feature vector of the input image is reconstructed using the principal component analysis transformation model to generate an image description vector.

[0046] Specifically, step S3 includes the following sub-steps:

[0047] S301: Obtain a high-dynamic visual scene training dataset that includes changes in lighting, viewpoint, objects, and seasons, with a balanced proportion of samples of each type of change. Ensure the size of the dataset and the proportion of each type of sample to ensure that the principal component analysis transformation model trained can fully fit the high-dynamic visual scene.

[0048] S302: For each image in the training dataset, extract its feature vector using the AlexC3 model built in step S2 to construct a training matrix X of size 64896*m, where 64896 represents the dimension of the feature vector extracted by the AlexC3 model, and m represents the number of images in the training dataset. The expression for X is as follows:

[0049] X = [x 12 … m ]

[0050] S303: Perform row-wise mean reduction on the training matrix X. The formula for mean reduction is as follows:

[0051]

[0052] In the formula, x i Let represent the i-th column (a 64896*1 dimensional vector) of the training matrix X, and m represent the number of images in the training dataset, i.e., the total number of columns in the training matrix. This represents the element in the i-th column of the training matrix after mean reduction.

[0053] S304: For the mean-reduced training matrix Calculate the corresponding covariance matrix C, and then calculate the eigenvalues ​​λ and eigenvectors v of this covariance matrix C. The formula for calculating the covariance matrix C is as follows:

[0054]

[0055]

[0056]

[0057] S305: Since the magnitude of the eigenvalues ​​of the covariance matrix corresponds to the magnitude of the variance contained in its eigenvectors, and a larger variance represents a denser information distribution along the direction of the eigenvector, this invention arranges the eigenvectors v in descending order according to the magnitude of the eigenvalue λ, and constructs a K*64896 transformation matrix P based on the transformed target dimension K. P can be expressed by the following formula:

[0058] P = [v λmax … λmin ]

[0059] In the formula, v λmax v is the eigenvector corresponding to the largest eigenvalue λmax. λmin λmin is the eigenvector corresponding to the minimum eigenvalue.

[0060] S306: A quantitative experiment was conducted to investigate the impact of the transformed target dimension K on the feature representation ability of the transformed new vector. The evaluation indicators were the information compression characteristics of the principal component vector space, the accuracy performance of the algorithm before and after transformation, and the running efficiency performance of the algorithm before and after transformation.

[0061] S307: Based on the quantitative experiments in S306, a comprehensive evaluation is conducted on the information compression characteristics based on principal component vector space, the accuracy performance of the algorithm before and after transformation, and the running efficiency performance of the algorithm before and after transformation to determine the optimal target dimension K. b And based on the optimal target dimension K b Generate the optimal transformation model P b .

[0062] S308: Transform the principal component analysis obtained from unsupervised learning into model P b This process is saved as a fixed step in the overall visual position recognition method. The process of using this feature vector to perform principal component transformation on the image feature vector x output by the AlexC3 model can be represented as follows:

[0063] y = x T ·b

[0064] In the formula, y represents the image description vector after principal component transformation.

[0065] For example, this invention uses the odometry branch of Kitty, a commonly used dataset in the field of visual position recognition, as the basis for the training set. It contains 22 color image sequences of forward-looking road conditions collected during vehicle movement, with a total mileage of 39.2 kilometers. The images in this dataset comprehensively reflect various changes that may occur in visual position recognition scenarios, making it an ideal source of training data. After certain screening, this invention uses 31,750 images from the sequences 00, 01, 02, 05, 08, 13, 15, 16, 19, and 21, which have higher scene complexity, as the final training dataset. After processing with the AlexC3 model, a trainable vector matrix X of size 64896*31750 is obtained.

[0066] Secondly, following steps S303, S304, and S305, multiple principal component analysis transformation models P with different target dimensions are trained. 256 P 512 P 1024 P 4096 P 8192 , where the subscript of P indicates the target dimension.

[0067] Subsequently, based on the evaluation metrics proposed in step S306—the information compression characteristics of the principal component vector space, the accuracy performance of the algorithm before and after transformation, and the running efficiency performance of the algorithm before and after transformation—quantitative performance evaluation experiments were conducted on principal component analysis transformation models with different target dimensions. The experimental results are as follows: Figure 3 , Figure 4 and Figure 5 As shown. Among them, in Figure 4 In the precision-recall curve, a larger area under the curve represents better accuracy performance; for example... Figure 3 As shown, the principal component vector space can retain most of the information of the original vector with a dimension number far lower than the original value (64896). Therefore, the vector after dimensionality reduction through principal component transformation has a stronger information-expressing ability for each dimension; at the same time, the large number of redundant dimensions removed makes the main direction of the vector more prominent, which is manifested as... Figure 4 The overall algorithm accuracy after principal component transformation is significantly higher than before transformation. Figure 4The different questions in the dataset represent different high-dynamic visual position recognition test datasets: Alderley, GardenspointsDay Left-Right, Gardenspoints Day Left-Night Right, and Nordland Summer-Winter. These test datasets are widely used in the field of visual position recognition for evaluating the accuracy and performance of algorithms. On the other hand, Figure 5 The efficiency experiments demonstrate that the low-dimensional vectors obtained by principal component transformation can significantly reduce the computation time of cosine similarity and improve the efficiency of the algorithm.

[0068] Finally, the results of the performance evaluation experiments are summarized: the information compression characteristics of the principal component space are concentrated in the first dimension, and the compression capability increases at a slower pace with the increase of the target dimension; the accuracy performance of the algorithm is positively correlated with the increase of the target dimension during principal component transformation, but the improvement in accuracy performance gradually decreases after the target dimension exceeds 1024; the running efficiency of the algorithm deteriorates with the increase of the target dimension, especially after the target dimension exceeds 1024. Therefore, considering the three evaluation indicators, this invention selects 1024 as the final target dimension of the unsupervised principal component analysis transformation model.

[0069] Step S4: Obtain the image description vector of the existing image from the database, calculate the similarity between the image description vector of the input image and the image description vector of the existing image; when the maximum similarity is greater than or equal to the similarity threshold, take the existing image corresponding to the maximum similarity as the similar image of the input image to obtain the visual position recognition result.

[0070] Specifically, step S4 includes the following sub-steps:

[0071] S401: Calculate the image description vector I of the input image. current Image description vector I of all existing images past The cosine similarity cs. The formula for calculating cs is as follows:

[0072]

[0073] In the formula, · represents the dot product of vectors, and ║║ represents the magnitude of the vector.

[0074] S402: Sort the cosine similarity to obtain the maximum similarity; compare the maximum similarity with the similarity threshold. If the maximum similarity is greater than or equal to the preset threshold, use the existing image corresponding to the maximum similarity as the similar image of the input image to obtain the visual position recognition result, that is, the input image has existing images with similar positions in the database; if the maximum similarity is less than the preset threshold, the current input image does not have existing images with similar positions in the database.

[0075] To further illustrate the beneficial effects of this invention, the following is a combination of... Figure 6 The accuracy and performance of the algorithm of this invention are demonstrated.

[0076] Figure 6 This paper presents a comparison of the accuracy performance of the algorithm of this invention with several state-of-the-art algorithms in the field of visual position recognition. The comparison methods include: DBoW3, an outstanding representative of traditional machine learning visual position recognition algorithms, currently the most widely used in practice; NetVLAD, which holds an important position in supervised learning-based deep learning visual position recognition algorithms and is one of the best performing visual position recognition algorithms to date; and Calc, a typical unsupervised learning-based visual position recognition algorithm that exhibits excellent accuracy in this category. The different questions in the figure represent different high-dynamic visual position recognition test datasets: Alderley, Gardenspoints Day Left-Right, Gardenspoints Day Left-Night Right, and Nordland Summer-Winter. These test datasets are widely used in the field of visual position recognition for evaluating algorithm accuracy performance.

[0077] Experimental results show that the accuracy performance of the algorithm in this invention significantly surpasses DBoW3 and Calc, and rivals NetVLAD. This result effectively demonstrates the effectiveness of the approach of using principal component transformation to perform unsupervised transfer learning on convolutional neural network models, and aligns with the original intention of this invention: to solve the problem of insufficient algorithm accuracy while ensuring low implementation cost through unsupervised learning, thus providing a new and feasible approach for the practical application of visual position recognition algorithms.

[0078] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the visual position recognition method as described above. Figure 7 The diagram shown is a hardware structure diagram of any device with data processing capabilities in which the visual position recognition method provided in the embodiments of the present invention is used, except... Figure 7 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0079] Accordingly, this application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the visual position recognition method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0080] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A visual position recognition method, characterized in that, The method includes: Obtain the input image; Use a convolutional neural network to extract feature vectors from the input image; Based on unsupervised learning, a principal component analysis transformation model is trained, and the feature vector of the input image is reconstructed using the principal component analysis transformation model to generate an image description vector. The image description vectors of existing images are obtained from the database. The similarity between the image description vector of the input image and the image description vectors of existing images is calculated. When the maximum similarity is greater than or equal to the similarity threshold, the existing image corresponding to the maximum similarity is used as the similar image of the input image to obtain the visual location recognition result. The process of generating an image description vector by reconstructing the feature vector of the input image using a principal component analysis transformation model trained through unsupervised learning includes: Obtain the training dataset; Extract the feature vector of each image in the training dataset to construct the training matrix X; The training matrix X is demeaned by row; For the mean-reduced training matrix Calculate the corresponding covariance matrix C, and calculate the eigenvalues ​​λ and eigenvectors v of the covariance matrix C; The eigenvectors v are sorted in descending order according to the magnitude of the eigenvalues ​​λ, and a transformation matrix P is constructed based on the transformed target dimension K. Based on a comprehensive evaluation of the information compression characteristics of the principal component vector space, the accuracy and performance of the algorithms before and after transformation, and the running efficiency of the algorithms before and after transformation, the optimal target dimension K is determined. b And based on the optimal target dimension K b Generate the optimal transformation model P b Principal component analysis transformation model P b ; The feature vector of the input image is reconstructed using the principal component analysis transformation model to generate an image description vector, as shown in the following expression: ； In the formula, x represents the image feature vector, and y represents the image description vector after principal component transformation.

2. The visual position recognition method according to claim 1, characterized in that, Acquiring the input image also includes: enhancing the contrast of the input image, converting the input image from the RGB color space to the YUV color space, and performing histogram equalization on the Y channel separately, then converting the input image back to the RGB color space and normalizing it; and standardizing the size of the input image based on the bicubic interpolation method.

3. The visual position recognition method according to claim 1, characterized in that, A convolutional neural network includes a first convolutional layer, a first ReLU layer, a first max pooling layer, a second convolutional layer, a second ReLU layer, a second max pooling layer, a third convolutional layer, and a Sigmoid layer connected in sequence.

4. The visual position recognition method according to claim 3, characterized in that, The construction process of a convolutional neural network is as follows: The convolutional neural network uses the AlexC3 model; Based on the structure and pre-training parameters of the general convolutional neural network model AlexNet; The first three convolutional layers of the general convolutional neural network model AlexNet are used as the main structure of the AlexC3 model, and the output is processed by a Sigmoid layer, replacing the max pooling and ReLU activation operations. Remove all fully connected layers from the general convolutional neural network model AlexNet; The number of convolutional kernels in the first and second convolutional layers is set to 64 and 192, respectively, while the number of convolutional kernels in the third convolutional layer remains unchanged.

5. The visual position recognition method according to claim 4, characterized in that, The expression for the Sigmoid layer is as follows: ； In the formula, This represents the value of the i-th element of the feature vector output by the third convolutional layer in the AlexC3 model. This represents the value of the i-th element of the feature vector output by the Sigmoid layer; The expression for the ReLU layer is as follows: ； In the formula, max() represents the function to find the maximum value. The input data represents the ReLU layer, and the output data represents the ReLU layer. The ReLU layer will output 0 for all input data less than or equal to 0, and leave the input data greater than 0 unchanged.

6. The visual position recognition method according to claim 1, characterized in that, The training dataset consists of highly dynamic visual scene images that simultaneously include variations in lighting, viewpoint, objects, and seasons, with a balanced proportion of samples representing each type of variation.

7. The visual position recognition method according to claim 1, characterized in that, Retrieve image description vectors of existing images from the database, and calculate the similarity between the image description vector of the input image and the image description vectors of the existing images; When the maximum similarity is greater than or equal to the similarity threshold, the existing image corresponding to the maximum similarity is used as a similar image to the input image to obtain the visual location recognition result, including: Calculate the image description vector I of the input image. current Image description vector I of all existing images past The cosine similarity cs; The cosine similarity scores are sorted to obtain the highest similarity score. The maximum similarity is compared with a similarity threshold. If the maximum similarity is greater than or equal to the preset threshold, the existing image corresponding to the maximum similarity is used as the similar image of the input image to obtain the visual location recognition result, that is, the input image has an existing image with a similar location in the database; if the maximum similarity is less than the preset threshold, the current input image does not have an existing image with a similar location in the database.

8. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the visual position recognition method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the visual position recognition method as described in any one of claims 1-7.

Images