Localization method and cluster center set generation method

Abstract

Disclosed in the present disclosure are a localization method and a cluster center set generation method. The localization method comprises: acquiring local image feature vectors of a query image of a target scene; acquiring a pre-generated cluster center set, and determining cluster centers to which the local image feature vectors belong and indexes of the cluster centers; deleting local image feature vectors of dynamic objects from the local image feature vectors; on the basis of retained local image feature vectors and cluster centers to which the retained local image feature vectors belong, generating global feature vectors; on the basis of the global feature vectors, acquiring, from pre-generated image localization fingerprint data, a sample image matching the query image and position information thereof; and on the basis of the position information of the matched sample image, determining a localization position of the query image. After local image feature vectors of a query image are acquired, local image feature vectors of dynamic objects are deleted, and localization is performed on the basis of local image feature vectors obtained after the dynamic objects are removed, thereby improving the accuracy of localizing scenes involving dynamic objects.

Description

Positioning method and method for generating cluster center set

[0001] The present disclosure claims priority to the Chinese patent application No. 202410851635.6, filed on June 27, 2024, and entitled "A positioning method and a method for generating a cluster center set", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of positioning, and in particular to a positioning method and a method for generating a cluster center set. BACKGROUND

[0003] Visual Positioning Recognition (VPR) is a technology for positioning using image information. VPR has broad application prospects in many fields, such as indoor positioning, indoor navigation, etc.

[0004] Due to the complex and changeable environment of real scenes, such as pedestrians, vehicles and various dynamic targets, dynamic targets will affect the accuracy of positioning results. How to effectively remove dynamic targets in images is one of the problems that VPR needs to solve.

[0005] Further, due to the influence of light, weather, shooting angle, etc. on images, and the lack of depth information in images taken by ordinary cameras / cameras, it is difficult for VPR to accurately label the true value, so in the absence of scene true value, generating an accurate scene feature representation is another problem that VPR needs to solve. SUMMARY

[0006] Therefore, the present disclosure provides a positioning method and a method for generating a cluster center set to improve the accuracy of positioning for scenes with dynamic targets.

[0007] In a first aspect, the present disclosure provides a positioning method, the method comprising:

[0008] obtaining an image local feature vector of a query image of a target scene, the query image recording dynamic targets and static targets in the target scene;

[0009] based on a cluster center set of the target scene generated in advance, obtaining a cluster center to which the image local feature vector belongs and an index of the cluster center;

[0010] deleting, from the image local feature vector, an image local feature vector of the dynamic target, the index of the cluster center to which the image local feature vector of the dynamic target belongs belonging to a dynamic target;

[0011] generate a global feature vector of the query image based on the reserved image local feature vector and a cluster center to which the reserved image local feature vector belongs;

[0012] obtain location information corresponding to a sample image matched with the query image from pre-generated image positioning fingerprint data based on the global feature vector of the query image;

[0013] determine a positioning location of the query image based on the location information corresponding to the matched sample image.

[0014] In a possible implementation, the obtaining of the image local feature vector of the query image of the target scene comprises:

[0015] inputting the query image into a pre-trained feature extraction model to obtain coordinates and feature vectors of key points of the query image, and coordinate regions and feature vectors of pixel blocks of the query image;

[0016] determining a pixel block to which the key point belongs based on the coordinates of the key points of the query image and the coordinate regions of the pixel blocks of the query image;

[0017] concatenating the feature vector of the key point of the query image with the feature vector of the pixel block to which the key point belongs to obtain the image local feature vector.

[0018] In a possible implementation, the feature extraction model comprises a key point feature extraction model and a pixel block feature extraction model, wherein,

[0019] the key point feature extraction model is configured to output the coordinates and the feature vectors of the key points of the query image;

[0020] the pixel block feature extraction model is configured to output the coordinate regions and the feature vectors of the pixel blocks of the query image.

[0021] In a possible implementation, the obtaining of the cluster center to which the image local feature vector belongs and the index of the cluster center based on the pre-generated cluster center set of the target scene comprises:

[0022] for each image local feature vector, obtaining a distance of the image local feature vector to a cluster center in the cluster center set;

[0023] taking the cluster center with the shortest distance as the cluster center to which the image local feature vector belongs, and obtaining the index of the cluster center from the cluster center set.

[0024] In a possible implementation, before concatenating the feature vector of the key point of the query image with the feature vector of the pixel block to which the key point belongs, the method further includes:

[0025] performing dimension reduction processing on the feature vector of the key point of the query image by using the pre-generated first principal component analysis dimension reduction parameter; and / or,

[0026] performing dimension reduction processing on the feature vector of the pixel block of the query image by using the pre-generated second principal component analysis dimension reduction parameter.

[0027] In a possible implementation, the generating the global feature vector of the query image based on the reserved image local feature vector and the cluster center to which the reserved image local feature vector belongs includes:

[0028] generating a local aggregated descriptor VLAD of the query image based on the reserved image local feature vector and the cluster center to which the reserved image local feature vector belongs;

[0029] normalizing the local aggregated descriptor VLAD of the query image to obtain the global feature vector of the query image.

[0030] In a possible implementation, the method further includes:

[0031] performing dimension reduction processing on the global feature vector of the query image by using the pre-generated third principal component analysis dimension reduction parameter.

[0032] In a second aspect, the present disclosure provides a method for generating a set of cluster centers, the method including:

[0033] obtaining image local feature vectors of sample images of a target scene, the sample images recording dynamic targets and static targets;

[0034] performing clustering on the image local feature vectors to obtain cluster centers and store the cluster centers in a dictionary, one cluster center corresponding to one index in the dictionary;

[0035] obtaining a mask of a dynamic target recorded by the sample images;

[0036] based on the mask of the dynamic target, determining cluster centers belonging to the dynamic target in the dictionary and marking the cluster centers in the dictionary.

[0037] In a possible implementation, the obtaining the mask of the dynamic target recorded by the sample images includes:

[0038] input the sample image into a pre-trained image segmentation model to obtain a segmentation mask corresponding to a plurality of key points in the sample image;

[0039] determine a mask of the dynamic target recorded in the sample image based on the category of the segmentation mask and the category of the dynamic target.

[0040] In a possible implementation, the determining of the cluster center belonging to the dynamic target in the dictionary based on the mask of the dynamic target comprises:

[0041] For any key point in the sample image, determine the cluster center to which the key point belongs;

[0042] determine a k-dimensional zero vector V based on the number k of cluster centers in the dictionary;

[0043] For any cluster center i, update the value of the element Vi in the vector V based on the number of dynamic targets included in the cluster center i, where i = 1, 2,..., k;

[0044] determine the cluster center corresponding to the preset number of elements as the cluster center belonging to the dynamic target based on the values of the elements in the updated vector V.

[0045] In a third aspect, the present disclosure provides a positioning device, which comprises:

[0046] a first obtaining unit configured to obtain an image local feature vector of a query image of a target scene, the query image recording dynamic targets and static targets in the target scene;

[0047] a first generating unit configured to obtain a cluster center to which the image local feature vector belongs and an index of the cluster center based on a cluster center set of the target scene generated in advance;

[0048] a deleting unit configured to delete, from the image local feature vector, an image local feature vector of the dynamic target, the index of the cluster center to which the image local feature vector of the dynamic target belongs belonging to dynamic targets;

[0049] a second generating unit configured to generate a global feature vector of the query image based on the retained image local feature vector and the cluster center to which the retained image local feature vector belongs;

[0050] a second obtaining unit configured to obtain, based on the global feature vector of the query image, position information corresponding to a sample image matching the query image in pre-generated image positioning fingerprint data;

[0051] The first determining unit is configured to determine the positioning position of the query image based on the position information corresponding to the matched sample image.

[0052] In a possible implementation, the first obtaining unit is specifically configured to input the query image into a pre-trained feature extraction model to obtain the coordinates and feature vectors of the key points of the query image, and the coordinates and feature vectors of the pixel blocks of the query image; determine the pixel block to which the key point belongs based on the coordinates of the key points of the query image and the coordinate regions of the pixel blocks of the query image; and splice the feature vectors of the key points of the query image and the feature vectors of the pixel blocks to which the key points belong to obtain the image local feature vectors.

[0053] In a possible implementation, the feature extraction model comprises a key point feature extraction model and a pixel block feature extraction model, wherein the key point feature extraction model is configured to output the coordinates and feature vectors of the key points of the query image, and the pixel block feature extraction model is configured to output the coordinate regions and feature vectors of the pixel blocks of the query image.

[0054] In a possible implementation, the first generating unit is specifically configured to, for each image local feature vector, obtain the distance from the image local feature vector to the cluster centers in the cluster center set; take the cluster center with the shortest distance as the cluster center to which the image local feature vector belongs, and obtain the index of the cluster center from the cluster center set.

[0055] In a possible implementation, the apparatus further comprises a dimension reduction processing unit, wherein before the feature vectors of the key points of the query image and the feature vectors of the pixel blocks to which the feature points belong are spliced, the dimension reduction processing unit is configured to perform dimension reduction processing on the feature vectors of the key points of the query image by using pre-generated first principal component analysis dimension reduction parameters, and / or perform dimension reduction processing on the feature vectors of the pixel blocks of the query image by using pre-generated second principal component analysis dimension reduction parameters.

[0056] In a possible implementation, the second generating unit is specifically configured to generate a local aggregate descriptor VLAD of the query image based on the retained image local feature vectors and the cluster centers to which the retained image local feature vectors belong; and perform normalization on the VLAD of the query image to obtain a global feature vector of the query image.

[0057] In a possible implementation, the dimension reduction processing unit is further configured to perform dimension reduction processing on the global feature vector of the query image by using pre-generated third principal component analysis dimension reduction parameters.

[0058] In a fourth aspect, the present disclosure provides a device for generating a set of cluster centers, the device comprising:

[0059] a third obtaining unit, configured to obtain image local feature vectors of sample images of a target scene, the sample images recording dynamic targets and static targets;

[0060] a clustering unit, configured to cluster the image local feature vectors to obtain cluster centers and store the cluster centers in a dictionary, one cluster center corresponding to one index in the dictionary;

[0061] a fourth obtaining unit, configured to obtain a mask of the dynamic targets recorded in the sample images;

[0062] a second determining unit, configured to determine, based on the mask of the dynamic targets, cluster centers belonging to the dynamic targets in the dictionary and mark the cluster centers in the dictionary.

[0063] In a possible implementation, the fourth obtaining unit is specifically configured to input the sample images into a pre-trained image segmentation model to obtain segmentation masks corresponding to a plurality of key points in the sample images; and determine the mask of the dynamic targets recorded in the sample images based on a category of the segmentation masks and a category of the dynamic targets.

[0064] In a possible implementation, the second determining unit is specifically configured to determine, for any key point in a sample image, a cluster center to which the key point belongs; determine a k-dimensional zero vector V based on a number k of cluster centers in the dictionary; update a value of an element Vi in the vector V based on a number of dynamic targets included in any cluster center i, where i = 1, 2,..., k; and determine, based on the values of the elements in the updated vector V, cluster centers corresponding to a preset number of elements as the cluster centers belonging to the dynamic targets.

[0065] In a fifth aspect, the present disclosure provides an electronic device, the device comprising: a memory and a processor;

[0066] the memory is configured to store related program codes;

[0067] the processor is configured to invoke the program codes to execute the method in any one of the implementation manners of the first aspect or the second aspect.

[0068] In a sixth aspect, the present disclosure provides a computer readable storage medium for storing a computer program, the computer program being configured to execute the method in any one of the implementation manners of the first aspect or the second aspect.

[0069] In a seventh aspect, the present disclosure provides a computer program product, which, when executed by a processor, implements the method according to any implementation manner of the first aspect or the second aspect.

[0070] Therefore, the present disclosure has the following beneficial effects:

[0071] In the above implementation manners of the present disclosure, in order to locate the target scene, the image local feature vector of the query image of the target scene can be acquired, wherein the query image records dynamic targets and static targets in the target scene. The dynamic targets can be pedestrians, vehicles, etc. A cluster center set of the target scene generated in advance can be acquired, the cluster center set includes a plurality of cluster centers, and each cluster center corresponds to an index. The cluster center to which the image local feature vector belongs and the index of the cluster center can be determined from the cluster center set. The image local feature vector of the dynamic target is deleted from the image local feature vector, wherein the index of the cluster center to which the local feature vector of the dynamic target belongs belongs to the dynamic target. The global feature vector of the query image is generated based on the retained image local feature vector and the cluster center to which the retained image local feature vector belongs. Based on the global feature vector of the query image, the sample image matched with the query image and the corresponding position information are acquired from the image location fingerprint data generated in advance, so that the location position of the query image can be determined based on the position information of the matched sample image, and the location of the target scene is realized. Through the method provided by the present disclosure, after the image local feature vector of the query image is acquired, the image local feature vector of the dynamic target is deleted, so that the location can be performed based on the image local feature vector after the dynamic target is removed, and the accuracy of the location of the scene including the dynamic target is improved. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments provided in the present disclosure, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0073] FIG. 1 is a flowchart of a positioning method provided by an embodiment of the present disclosure;

[0074] FIG. 2 is a schematic diagram of a positioning method provided by an embodiment of the present disclosure;

[0075] FIG. 3 is a flowchart of a method for generating a cluster center set provided by an embodiment of the present disclosure;

[0076] FIG. 4 is a schematic diagram of a positioning device provided by an embodiment of the present disclosure;

[0077] FIG. 5 is a schematic diagram of an apparatus for generating a set of cluster centers according to an embodiment of the present disclosure;

[0078] FIG. 6 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present disclosure. The described embodiments are only exemplary implementations of the present disclosure, and not all implementations. Those skilled in the art can obtain other embodiments without creative labor by combining the embodiments of the present disclosure, and these embodiments are also within the protection scope of the present disclosure.

[0080] Visual Positioning Recognition (VPR) is a technology for positioning using image information. Visual Positioning Recognition has broad application prospects in many fields, such as indoor positioning, indoor navigation, etc.

[0081] Due to the complex and changeable environment of real scenes, such as pedestrians, vehicles, robots and various dynamic targets, dynamic targets will affect the accuracy of positioning results. How to effectively remove dynamic targets in images is one of the problems that VPR needs to solve.

[0082] Further, due to the influence of light, weather, shooting angle, etc. on images and the lack of depth information in images taken by ordinary cameras / cameras, it is difficult for VPR to accurately label the true value, so generating an accurate scene feature representation without scene true value is another problem that VPR needs to solve.

[0083] Based on this, the embodiment of the present disclosure provides a positioning method to improve the accuracy of positioning a scene including a dynamic target. In specific implementation, in order to position a target scene, an image local feature vector of a query image of the target scene can be obtained, wherein the query image records a dynamic target and a static target in the target scene. The dynamic target can be a pedestrian, a vehicle, a robot, etc. A cluster center set of the target scene pre-generated can be obtained, the cluster center set includes a plurality of cluster centers, and each cluster center corresponds to an index. The cluster center to which the image local feature vector belongs and the index of the cluster center can be determined from the cluster center set. The image local feature vector of the dynamic target is deleted from the image local feature vector, wherein the index of the cluster center to which the image local feature vector of the dynamic target belongs belongs to the dynamic target. A global feature vector of the query image is generated based on the retained image local feature vector and the cluster center to which the retained image local feature vector belongs. Based on the global feature vector of the query image, a sample image matched with the query image and corresponding position information are obtained from the pre-generated image positioning fingerprint data, so that the positioning position of the query image can be determined based on the position information of the matched sample image, and the positioning of the target scene is realized. Through the method provided by the present disclosure, after obtaining the image local feature vector of the query image, the image local feature vector of the dynamic target is deleted, so that the positioning can be performed based on the image local feature vector after removing the dynamic target, and the accuracy of positioning the scene including the dynamic target is improved.

[0084] In order to facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, the following will be specifically introduced in conjunction with the drawings.

[0085] Referring to FIG. 1, FIG. 1 is a flowchart of a positioning method provided by the embodiment of the present disclosure.

[0086] The method can be executed by a processing device, wherein the processing device can be a terminal or a server. The method can include the following steps:

[0087] S101: Obtain an image local feature vector of a query image of a target scene.

[0088] The query image records a dynamic target and a static target in the target scene. The dynamic target represents a target category that needs to be removed when positioning the query image. For example, the dynamic target can be a pedestrian, a vehicle, a robot, etc. The image local feature vector of the query image can be obtained by feature extraction on the query image.

[0089] In a possible implementation, the image local feature vector of the query image can be extracted by a pre-trained feature extraction model. In order to enable the feature extraction model to more accurately extract the image features of the query image, the feature extraction model can be used to extract feature vectors of different dimensions of the query image, and then the different feature vectors are spliced to obtain the image local feature vector.

[0090] In a specific implementation, the query image is input into the pre-trained feature extraction model to obtain the coordinates of the key points of the query image and the feature vectors of the key points, and the coordinate region of the pixel block of the query image and the feature vector of the pixel block. In the query image, a plurality of key points can be extracted, or a plurality of pixel blocks can be extracted, that is, the feature extraction model can extract feature vectors of different dimensions of the key points and the pixel blocks of the query image. Based on the coordinates of the key points in the query image and the coordinate region of the pixel block of the query image, the pixel block to which each key point belongs can be determined. The feature vector of each key point of the query image is spliced with the feature vector of the pixel block to which the key point belongs to obtain the image local feature vector.

[0091] In a possible implementation, the feature extraction model comprises:

[0092] a key point feature extraction model and a pixel block feature extraction model, wherein the key point feature extraction model is configured to output the coordinates of a plurality of key points of the query image and the feature vector of each key point;

[0093] the pixel block feature extraction model is configured to output the coordinate region of a plurality of pixel blocks of the query image and the feature vector of each pixel block.

[0094] Optionally, the key point feature extraction model can be a SuperPoint network, and the feature vector of the key point output by the SuperPoint network can be represented as M*C sp , wherein M represents the number of key points, C sp represents the image feature vector corresponding to each key point, and the coordinates (x i , y i ) corresponding to each key point can also be obtained.

[0095] The pixel block feature extraction model can also be a DINOv2 network. DINOv2 is a feature extraction network that is self-supervised pre-trained on a super large-scale image dataset, has strong representation and discrimination ability, and can provide strong semantic representation for downstream tasks. The DINOv2 network can take the pixel block r*r of the query image as input, calculate the feature vector, and output the feature vector of the query image. For example, when the query image takes N pixel blocks of r*r as the input of the DINOv2 feature extraction network, N C dino dimensional feature vectors can be output, N represents the number of pixel blocks, C dino represents the image feature vector corresponding to each pixel block, and the coordinate region of each pixel block can be represented as [X i ]*[Y i ].

[0096] When the feature vectors of multiple key points and the feature vectors of multiple pixel blocks of the query image are obtained, the pixel block to which each key point belongs can be determined based on the coordinates of the key points and the coordinate regions of the pixel blocks. For example, when the coordinates of the key points are represented as (x i , y i ), the coordinate region of the pixel block to which the key point belongs can be represented as Then, the feature vector of each key point of the query image is spliced with the feature vector of the pixel block to which the key point belongs, to obtain the spliced image local feature vector corresponding to the key point. The above operation is performed for multiple key points to obtain the spliced image local feature vectors corresponding to the multiple key points and combine them, that is, the image local feature vector of the query image is determined.

[0097] By using different feature extraction networks to extract different image features of the query image and splicing the different image features obtained, the image features of the query image extracted by the feature extraction model have stronger semantic representation, which can more accurately distinguish dynamic targets, so as to improve the accuracy of positioning the query image.

[0098] In one possible implementation, to avoid the dimension of the image local feature vector obtained by splicing being too large, resulting in excessive calculation in the subsequent image processing link, the feature vector extracted by the feature extraction network can be processed by dimension reduction. The specific implementation of dimension reduction processing can include:

[0099] The feature vector of the key point of the query image is processed by the pre-generated first principal component analysis dimension reduction parameter. That is, the feature vector of the key point is multiplied by the first principal component analysis dimension reduction parameter to obtain the dimension-reduced feature vector of the key point. And / or, the feature vector of the pixel block of the query image is processed by the pre-generated second principal component analysis dimension reduction parameter. That is, the feature vector of the pixel block is multiplied by the second principal component analysis dimension reduction parameter to obtain the dimension-reduced feature vector of the pixel block.

[0100] That is, the dimension reduction processing can be performed only by the pre-generated first principal component analysis dimension reduction parameter on the feature vector of the key point of the query image to obtain the dimension-reduced feature vector of the key point. Or, the dimension reduction processing can also be performed only by the pre-generated second principal component analysis dimension reduction parameter on the feature vector of the pixel block of the query image to obtain the dimension-reduced feature vector of the pixel block. Or, both the pre-generated first principal component analysis dimension reduction parameter and the pre-generated second principal component analysis dimension reduction parameter are used to process the feature vector of the key point of the query image and the feature vector of the pixel block of the query image to obtain the dimension-reduced feature vector of the key point and the dimension-reduced feature vector of the pixel block. The process of determining the first principal component analysis dimension reduction parameter and the second principal component analysis dimension reduction parameter can be referred to in subsequent embodiments, which will not be described here.

[0101] Further, the dimension-reduced feature vector of the key point and / or the dimension-reduced feature vector of the pixel block can also be normalized to improve the efficiency of image processing.

[0102] For example, the L2 norm normalization method can be used to normalize the dimension-reduced feature vector of the key point. The L2 norm of the dimension-reduced feature vector of the key point can be represented as the square sum of each element in the feature vector, and the square root of the square sum is the L2 norm. Each element in the dimension-reduced feature vector of the key point is divided by the L2 norm to obtain the normalized dimension-reduced feature vector of the key point.

[0103] Based on this, the feature vector of the key point (after dimension reduction and / or normalization) and the feature vector of the pixel block (after dimension reduction and / or normalization) can be spliced to obtain the image local feature vector of the query image. The image local feature vector is spliced from the feature vectors extracted by the two feature extraction networks, which can more accurately express the features of the image and has a stronger semantic representation.

[0104] In a possible implementation, the local feature vector of the spliced image can also be processed by the pre-generated fourth principal component analysis dimension reduction parameter to reduce the dimension of the local feature vector of the image, thereby facilitating subsequent data processing. Details of the process of determining the fourth principal component analysis dimension reduction parameter can be found in subsequent embodiments and are not described here.

[0105] S102: Obtain a cluster center to which the local feature vector of the image belongs and an index of the cluster center based on the pre-generated cluster center set of the target scene.

[0106] The cluster center set is pre-generated and stored in the processing device, and includes a plurality of cluster centers, each of which corresponds to an index. The cluster center set includes a cluster center associated with a non-dynamic target and / or a cluster center associated with a dynamic target. The index of the cluster center can be used to distinguish whether the cluster center is associated with a dynamic target. That is, the index of the cluster center to which the local feature vector of the image belongs can be calculated to delete the dynamic target. Details of the process of determining the cluster center set can be found in subsequent embodiments and are not described here.

[0107] As can be seen from the above embodiments, the local feature vector of the query image is composed of the local feature vectors of a plurality of key points. Therefore, when calculating the cluster center to which the local feature vector of the image belongs, the local feature vector of each key point can be determined according to the similarity between the local feature vector and a plurality of cluster centers in the cluster set. For example, the distance between the local feature vector and a plurality of cluster centers in the cluster set can be obtained. The smaller the distance, the higher the similarity between the local feature vector and the cluster center. Therefore, the cluster center to which the local feature vector (key point) belongs can be determined by determining the shortest Euclidean distance between the local feature vector and a plurality of cluster centers. Since each cluster center in the cluster center set has an index, the index of the cluster center to which the local feature vector belongs can be obtained.

[0108] S103: Delete the local feature vector of the dynamic target from the local feature vector of the image.

[0109] Since the dynamic target and the static target in the target scene are recorded in the query image, in order to improve the accuracy of positioning the target scene and reduce the influence of the dynamic target, the local feature vector corresponding to the dynamic target in the query image can be deleted after the local feature vector of the query image is generated. The index of the cluster center to which the local feature vector of the dynamic target belongs belongs to the dynamic target.

[0110] By determining the cluster center to which the image local feature vector of each key point belongs and the index of each cluster center through the above steps, the key points belonging to the dynamic target can be determined according to the index of the cluster center associated with the dynamic target, and then the image local feature vector of the dynamic target is deleted.

[0111] S104: Generating the global feature vector of the query image based on the reserved image local feature vector and the cluster center to which the reserved image local feature vector belongs.

[0112] The global feature vector of the query image can be calculated by using the cluster center to which the reserved image local feature vector in the cluster center set belongs and the reserved image local feature vector.

[0113] In a possible implementation, the local aggregated descriptor (Vector of Locally Aggregated Descriptors, VLAD) of the query image can be generated based on the reserved image local feature vector and the cluster center to which the reserved image local feature vector belongs. The VLAD of the query image is normalized to obtain the global feature vector of the query image.

[0114] In a specific implementation, the local aggregated descriptor VLAD of the query image can be calculated in the following manner:

[0115] wherein x i represents the key point (reserved key point) of the non-dynamic target in the query image, Cx i represents the image local feature vector of the reserved key point in the query image, n represents the number of the reserved image local feature vectors, C k represents the cluster center to which the reserved image local feature vector belongs, and k represents the number of the cluster centers to which the reserved image local feature vector belongs. When the key point x i belongs to the cluster center C k , the value of a k (x i ) is 1; when the key point x i does not belong to the cluster center C k , the value of a k (x i ) is 0. Through the above calculation method, the local aggregated descriptor VLAD of the query image can be obtained.

[0116] The local aggregation descriptor VLAD is then normalized to obtain a global feature vector of the query image. For example, the local aggregation descriptor VLAD can be internally normalized and globally normalized, where the internal normalization refers to normalizing the feature vector of each cluster center dimension, and the global normalization refers to normalizing the one-dimensional vector obtained by concatenating the internally normalized feature vectors, thereby obtaining the global feature vector of the query image.

[0117] In a possible implementation, to reduce the dimension of the image feature and simplify the data processing process, the global feature vector of the query image can also be processed by a third principal component analysis dimension reduction parameter generated in advance to obtain a global feature vector after dimension reduction. The process of determining the third principal component analysis dimension reduction parameter can be referred to in subsequent embodiments and will not be described here.

[0118] S105: Based on the global feature vector of the query image, location information corresponding to a sample image matching the query image is obtained from the pre-generated image positioning fingerprint data.

[0119] The pre-generated image positioning fingerprint data includes a plurality of sample images of the pre-generated target scene, and each sample image includes pre-determined location information. The sample image matching the query image, that is, the sample image with the highest similarity to the query image, can be determined in the image positioning fingerprint data, and the location information of the matching sample image is obtained to determine the positioning of the query image according to the location information of the matching sample image.

[0120] Optionally, each sample image in the image positioning fingerprint data corresponds to a preset feature vector. The process of determining the preset feature vector can be referred to in subsequent embodiments and will not be described here. By matching the global feature vector of the query image with the preset feature vectors of the plurality of sample images, the preset feature vector with the highest similarity to the global feature vector can be determined, and the sample image corresponding to the preset feature vector is the sample image that matches the query image most. Specifically, the reciprocal of the Euclidean distance between the global feature vector and the plurality of preset feature vectors can be calculated as the similarity, that is, the smaller the Euclidean distance, the greater the similarity, and the higher the matching degree of the query image and the sample image.

[0121] S106: Based on the location information of the matching sample image, the positioning of the query image is determined.

[0122] Since the sample images all include location information, after the sample image matching the query image is determined, the positioning of the query image can be determined according to the location information of the matching sample image.

[0123] By means of the positioning method provided in the embodiments of the present disclosure, after the image local feature vector of the query image is obtained, the image local feature vector of the dynamic target can be deleted, so that positioning can be performed based on the image local feature vector after the dynamic target is removed, and the accuracy of positioning for a scene including a dynamic target is improved. Moreover, different dimensions of image features of the query image can be extracted by means of the feature extraction network, and more accurate feature representation of the query image can be obtained.

[0124] In the above embodiments of the present disclosure, the process of positioning the query image can be an online processing process. When the query image is positioned online, the data determined in advance in the offline processing process can be called for processing, including the pre-generated cluster center set, image positioning fingerprint data, principal component analysis dimension reduction parameters, and the like.

[0125] In order to clearly understand the working principle of the positioning method provided in the embodiments of the present disclosure, the offline processing process will be introduced below.

[0126] In specific implementation, a plurality of sample images of a target scene can be obtained, and the plurality of sample images can be images of different perspectives and different ranges taken for the target scene. For any sample image, an image local feature vector of the sample image, also referred to as a sample image feature vector, can be obtained. For example, the sample image can be input into a feature extraction model for processing. That is, a plurality of sample image feature vectors of the plurality of sample images can be obtained by means of the feature extraction model.

[0127] In a possible implementation, the sample image feature vector of the sample image can be extracted by means of a pre-trained feature extraction model. In order to enable the feature extraction model to more accurately extract the image features of the sample image, different dimension feature vectors of the sample image can be extracted by means of the feature extraction model, and then the different feature vectors can be spliced to obtain the sample image feature vector.

[0128] In specific implementation, the sample image is input into the pre-trained feature extraction model to obtain the coordinates of the key points of the sample image and the feature vectors of the key points, and the coordinate region of the pixel block of the sample image and the feature vectors of the pixel block. In the sample image, a plurality of key points can be extracted, or a plurality of pixel blocks can be extracted, that is, the feature extraction model can extract feature vectors of different dimensions of the key points and the pixel blocks of the sample image. Based on the coordinates of the key points in the sample image and the coordinate region of the pixel block of the sample image, the pixel block to which each key point belongs can be determined. The feature vector of each key point of the sample image is spliced with the feature vector of the pixel block to which the key point belongs to obtain the sample image feature vector.

[0129] The specific implementation principle of the feature extraction model can be referred to the introduction process of the image local feature vector of the query image in the above embodiment, and will not be described here.

[0130] In a possible implementation, to avoid the dimension of the feature vector of the spliced sample image being too large and simplify the data processing of the image, the feature vector extracted by the feature extraction network can be processed in dimension reduction.

[0131] In a possible implementation, the principal component analysis method can be used to process the feature vector of the key point of the sample image and the feature vector of the pixel block of the sample image in dimension reduction. The principal component analysis method is a multivariate statistical method that uses dimension reduction to transform a plurality of original indexes (feature vectors) into a few comprehensive indexes, and the comprehensive indexes are principal components. The principal component analysis method can be used to reduce the dimension of data while maintaining the features with the largest variance contribution in the data set. In this embodiment, the dimension of the feature vector after dimension reduction, that is, the number of principal components. The feature vector after dimension reduction can be determined by setting the cumulative contribution rate of the principal components to the variance of the original feature vector. For example, in this embodiment, the cumulative contribution rate of the principal components can be set to 80%, so that the feature vector after dimension reduction can be obtained.

[0132] Taking the feature vector of the key point of the sample image as an example, the feature vectors of the key points of the sample image are standardized in specific implementation. The correlation coefficients between the feature vectors of the standardized key points are calculated to reflect the correlation between the feature vectors of the key points. The covariance matrix is calculated based on the feature vectors of the standardized key points, and the eigenvalues and eigenvectors of the covariance matrix are calculated. The eigenvalue represents the variance of the principal component, and the eigenvector represents the direction of the principal component. According to the size of the eigenvalue, the eigenvectors corresponding to the eigenvalues with a cumulative variance contribution rate of 80% are retained. The principal components are synthesized by using the retained eigenvectors. The principal component is a linear combination of the feature vectors of the original key points, and the coefficients are the eigenvectors corresponding to the eigenvalues. That is, the coefficients of the principal component are the first principal component analysis dimension reduction parameters for dimension reduction processing of the feature vectors of the key points of the sample image.

[0133] Based on the same principle, the second principal component analysis dimension reduction parameter for dimension reduction processing of the feature vector of the pixel block of the sample image can also be obtained. The first principal component analysis dimension reduction parameter and the second principal component analysis dimension reduction parameter obtained in advance can be stored, so that when the query image is processed online for positioning in the future, the corresponding principal component analysis dimension reduction parameter stored in advance can be directly called for dimension reduction processing, and the efficiency of data processing is improved.

[0134] It should be noted that the principal component analysis method provided in the above embodiments for dimensionality reduction processing is only an exemplary description, and is not limited to the above implementation manner. Other dimensionality reduction processing algorithms that can achieve similar effects also belong to the protection scope of the present disclosure.

[0135] Optionally, the feature vectors of the key points after dimensionality reduction and / or the feature vectors of the pixel blocks after dimensionality reduction can also be normalized to simplify subsequent image data processing. For example, the L2 norm normalization method can be used for normalization.

[0136] Based on this, the feature vectors of the key points (after dimensionality reduction and / or normalization) and the feature vectors of the pixel blocks (after dimensionality reduction and / or normalization) can be spliced to obtain the image local feature vectors of the sample image. Then, the principal component analysis method is used to perform dimensionality reduction processing on the image local feature vectors of the sample image to obtain fourth principal component analysis dimensionality reduction parameters.

[0137] In a possible implementation manner, after obtaining the image local feature vectors corresponding to each sample image, the image local feature vectors of the plurality of sample images can be clustered to determine a plurality of cluster centers and store them in a dictionary, thereby obtaining a cluster center set composed of the plurality of cluster centers. In order to distinguish different cluster centers conveniently, a unique corresponding index can be set for each cluster center, so as to facilitate determination of the corresponding cluster center according to the index. For example, when k cluster centers are included, the index corresponding to each cluster center can be set as i, i = 1, 2,..., k.

[0138] Optionally, the K-means clustering algorithm can be used to cluster the plurality of image local feature vectors. The K-means clustering algorithm is an iterative solution clustering analysis algorithm, which can divide the plurality of image local feature vectors into K groups. K image local feature vectors are randomly selected as initial cluster centers. Then, the distance between each image local feature vector and each seed cluster center is calculated, and each image local feature vector is assigned to the nearest cluster center. The cluster center and the image local feature vectors assigned to the cluster center represent a cluster. Each time an image local feature vector is assigned, the cluster center is recalculated according to the image local feature vectors included in the current cluster. For example, the average value of the image local feature vectors included in the current cluster can be calculated as a new cluster center. This clustering process will be repeated until a certain termination condition is met. For example, the termination condition can be that no (or less than a preset number of) image local feature vectors are reassigned to different clusters.

[0139] It should be noted that the number of cluster centers provided in the above embodiments can be set according to actual demand scenarios, for example, can be determined in combination with the number of sample images, and the present disclosure embodiments do not limit this.

[0140] In order to more accurately realize the positioning of the sample image, a dynamic target included in the sample image can be removed. For example, the dynamic target can be a pedestrian, a vehicle, a robot, etc. In a possible implementation, for any sample image of a target scene, the sample image is input into an image segmentation model to obtain a segmentation mask corresponding to each key point in the sample image. The key point can be determined according to the feature extraction model provided in the above embodiments. The image segmentation model can be a pre-trained model, which can distinguish and gather pixel points of the same semantics according to different semantic information, so as to distinguish different parts of the sample image. Each part can be marked with a segmentation mask and a corresponding class label.

[0141] Further, according to the class of the segmentation mask of each part of the sample image output by the image segmentation model and the class of the dynamic target, a dynamic target mask corresponding to each key point can be determined, which can indicate whether the key point belongs to the dynamic target. For example, the mask belonging to the dynamic target class can be set to 1, and the mask not belonging to the dynamic target class can be set to 0, so that the dynamic target mask corresponding to each key point can be determined according to the segmentation mask of the sample image and the corresponding class label. That is, when the dynamic target mask corresponding to the key point is 1, it indicates that the key point belongs to the dynamic target; when the dynamic target mask corresponding to the key point is 0, it indicates that the key point does not belong to the dynamic target.

[0142] For the image local feature vector corresponding to any key point in the sample image, the cluster center to which the key point belongs can be determined according to the similarity between the image local feature vector and a plurality of cluster centers in the cluster set. For example, the similarity can be represented as the distance between the image local feature vector and the cluster center. When the distance is the smallest, it indicates that the similarity between the image local feature vector and the cluster center is the highest, that is, the cluster center to which the key point belongs is determined. In this way, the cluster centers to which a plurality of key points in the sample image belong can be determined, that is, the cluster centers to which each key point in a plurality of sample images of the target scene belongs are determined. Then, the dynamic target masks of the key points included in each cluster center are counted to determine the cluster center associated with the dynamic target.

[0143] In a possible implementation, the cluster center associated with the dynamic target can be determined in the following manner. Based on the number K of the plurality of cluster centers in the cluster set, a K-dimensional zero vector V is determined. For an index i of any cluster center, the value of an element Vi in the vector V is updated based on the dynamic target mask of the key point included in the cluster center i, where i = 1, 2, …, k. That is, after the plurality of key points included in the cluster center i is determined, the dynamic target mask corresponding to each key point is determined, and then the dynamic target masks of the key points are added to obtain the value of the element Vi corresponding to the cluster center i. When the dynamic target mask of the key point belonging to the dynamic target is 1, the value of the element Vi can represent the number of dynamic targets included in the current cluster center. The greater the value of the element Vi, the more dynamic targets the current cluster center includes.

[0144] Based on the values of the elements in the updated vector V, the cluster centers corresponding to the preset number of elements are determined as the cluster centers associated with the dynamic target. For example, the values of the elements can be sorted in descending order, and then the cluster centers corresponding to the first preset number of elements are selected as the cluster centers associated with the dynamic target. That is, the first preset number of cluster centers including more dynamic targets are selected.

[0145] Based on this, after the image local feature vector of the query image is processed online, the cluster center to which the image local feature vector of each key point of the query image belongs can be calculated, and the index of the cluster center to which each key point belongs is determined, so that whether the cluster center to which the key point belongs is the cluster center associated with the dynamic target can be determined.

[0146] When the cluster center set of the sample image and the cluster center associated with the dynamic target are determined, the cluster center associated with the dynamic target can be deleted from the cluster center set, and based on the deletion of the cluster center associated with the dynamic target and the deletion of the image local feature vector of the dynamic target, the preset feature vector of the sample image is determined.

[0147] In a possible implementation, the local aggregate descriptor VLAD of the sample image can be generated based on the deletion of the cluster center associated with the dynamic target and the deletion of the image local feature vector of the dynamic target. Then, the local aggregate descriptor VLAD is normalized to obtain the preset feature vector of the sample image.

[0148] In specific implementation, the local aggregate descriptor VLAD of the sample image can be calculated in the following manner: wherein x i represents the key point of the non-dynamic target in the sample image, Cx iLet C represent the local feature vectors retained in the sample image, where n represents the number of retained local feature vectors. k This represents the cluster center to which the preserved local feature vectors of the image belong, and k represents the number of cluster centers to which the preserved local feature vectors of the image belong. When the keypoint x i Belongs to cluster center C k At that time, a k (x i The value of ) is 1; when the key point x i Not belonging to cluster center C k At that time, a k (x i The value of ) is 0. Using the above calculation method, the Local Accumulation Descriptor (VLAD) of the query image can be obtained.

[0149] Then, the Locally Accumulated Descriptor (VLAD) is normalized to obtain the preset feature vector of the sample image. For example, the VLAD can be normalized internally and globally. Internal normalization means normalizing the feature vector of each cluster center dimension, while global normalization means concatenating the internally normalized feature vectors into a one-dimensional vector and then normalizing that one-dimensional vector to obtain the preset feature vector of the sample image.

[0150] In one possible implementation, in order to reduce the dimensionality of image features and simplify the data processing, principal component analysis can be used to reduce the dimensionality of the preset feature vectors of the sample images, obtain the third principal component analysis dimensionality reduction parameters, and store them in the processing device.

[0151] The above embodiments describe an offline processing procedure for determining the preset feature vectors of sample images. The processing device can store the generated data, including cluster center sets, principal component analysis dimensionality reduction parameters, and the preset feature vectors of the sample images. This allows for direct access to the pre-stored data during subsequent online image localization, reducing localization time and improving efficiency.

[0152] The principle of the positioning method will be introduced below in conjunction with a specific application scenario.

[0153] Referring to Figure 2, Figure 2 is a schematic diagram of a positioning method provided by an embodiment of this disclosure.

[0154] The query image is input into the key point feature extraction model to obtain a feature vector of the key point. The query image is also input into the pixel block feature extraction model to obtain a feature vector of the pixel block. After obtaining the feature vector of the key point and the feature vector of the pixel block, the feature vector of the key point and the feature vector of the pixel block can be spliced to obtain an image local feature vector. Based on the image local feature vector and a pre-stored cluster center set, VLAD calculation is performed to obtain a global feature vector after removing a dynamic target. After determining the global feature vector of the query image, the global feature vector can be matched with pre-set feature vectors of a plurality of sample images to determine position information of a sample image matched with the query image, and a positioning position of the query image is determined based on the position information.

[0155] Based on the above method embodiment, the embodiment of the disclosure further provides a method for generating a cluster center set. Referring to FIG. 3, a flowchart of a method for generating a cluster center set according to an embodiment of the disclosure is shown.

[0156] The method can include the following steps:

[0157] S301: Obtain an image local feature vector of a sample image of a target scene, and the sample image records a dynamic target and a static target.

[0158] The dynamic target can represent a target category that needs to be removed when positioning the sample image. For example, the dynamic target can be a pedestrian, a vehicle, a robot, etc.

[0159] S302: Cluster the image local feature vector to obtain a cluster center and store it in a dictionary.

[0160] For example, the K-means clustering algorithm can be used to cluster a plurality of image local feature vectors to obtain a plurality of cluster centers and form a dictionary. One cluster center in the dictionary can correspond to a unique index. It should be noted that the dictionary is a way to record the cluster center, that is, the dictionary is equivalent to the cluster center set. In addition, a separate dictionary can be established for each scene in general. For example, the position of the object to be positioned in a shopping mall is determined by VPR, and then the internal images of the shopping mall can be pre-collected to establish a corresponding dictionary.

[0161] S303: Obtain a mask of the dynamic target recorded by the sample image.

[0162] In one possible implementation, the sample image can be input into a pre-trained image segmentation model to obtain a segmentation mask corresponding to a plurality of key points in the sample image.

[0163] Based on the category of the segmentation mask and the category of the dynamic target, a mask of the dynamic target of the sample image record is determined. That is, the key points belonging to the dynamic target category are determined according to the category of the segmentation mask, and the mask belonging to the dynamic target in the sample image can be determined.

[0164] S304: Based on the mask of the dynamic target, the cluster centers belonging to the dynamic target in the dictionary are determined and marked in the dictionary.

[0165] The cluster center to which each key point belongs can be determined according to the distance between the image local feature vector of the key point and each cluster center in the dictionary. Then, based on the mask of the dynamic target corresponding to the key point, the cluster center belonging to the dynamic target is determined and marked.

[0166] Optionally, the first preset number of cluster centers including a larger number of dynamic targets can be selected as the cluster centers belonging to the dynamic target. In a specific implementation, for any key point in the sample image, the cluster center to which the key point belongs is determined; based on the number k of cluster centers in the dictionary, a k-dimensional zero vector V is determined; for any cluster center i, based on the number of dynamic targets included in the cluster center i, the value of the element Vi in the vector V is updated, where i = 1, 2,..., k; based on the values of the elements in the updated vector V, the cluster centers corresponding to the preset number of elements are determined as the cluster centers belonging to the dynamic target.

[0167] It should be noted that only the principle of generating the cluster center set is introduced in this embodiment, and the specific implementation process can be referred to the above method embodiment, and the same parts will not be described here.

[0168] Based on the above method embodiment, the present embodiment also provides a positioning device. Referring to FIG. 4, it is a schematic diagram of a positioning device provided by the present embodiment.

[0169] The device 400 includes:

[0170] The first acquisition unit 401 is configured to acquire an image local feature vector of a query image of a target scene, the query image recording dynamic targets and static targets in the target scene;

[0171] The first generation unit 402 is configured to obtain a cluster center to which the image local feature vector belongs and an index of the cluster center based on a cluster center set of the target scene generated in advance;

[0172] The deletion unit 403 is configured to delete, from the image local feature vector, an image local feature vector of the dynamic target, the index of the cluster center to which the image local feature vector of the dynamic target belongs belonging to the dynamic target;

[0173] The second generation unit 404 is configured to generate a global feature vector of the query image based on the reserved image local feature vector and a cluster center to which the reserved image local feature vector belongs.

[0174] The second acquisition unit 405 is configured to acquire, based on the global feature vector of the query image, position information corresponding to a sample image matched with the query image from pre-generated image positioning fingerprint data.

[0175] The first determination unit 406 is configured to determine a positioning position of the query image based on the position information of the matched sample image.

[0176] In a possible implementation, the first acquisition unit 401 is specifically configured to input the query image into a pre-trained feature extraction model, acquire coordinates and feature vectors of key points of the query image, and acquire coordinates and feature vectors of a pixel block of the query image; determine a pixel block to which the key points belong based on the coordinates of the key points of the query image and the coordinates of the pixel block of the query image; and splice the feature vectors of the key points of the query image and the feature vectors of the pixel block to which the key points belong, to obtain the image local feature vector.

[0177] In a possible implementation, the feature extraction model comprises a key point feature extraction model and a pixel block feature extraction model, wherein the key point feature extraction model is configured to output coordinates and feature vectors of key points of the query image; and the pixel block feature extraction model is configured to output coordinates and feature vectors of a pixel block of the query image.

[0178] In a possible implementation, the first generation unit 402 is specifically configured to acquire, for each image local feature vector, a distance from the image local feature vector to a cluster center in the cluster center set; take a cluster center with the shortest distance as a cluster center to which the image local feature vector belongs, and acquire an index of the cluster center from the cluster center set.

[0179] In a possible implementation, the apparatus further comprises a dimension reduction processing unit; before splicing the feature vectors of the key points of the query image and the feature vectors of the pixel block to which the feature points belong, the dimension reduction processing unit is configured to perform dimension reduction processing on the feature vectors of the key points of the query image by using pre-generated first principal component analysis dimension reduction parameters; and / or perform dimension reduction processing on the feature vectors of the pixel block of the query image by using pre-generated second principal component analysis dimension reduction parameters.

[0180] In a possible implementation, the second generation unit 404 is specifically configured to generate a local aggregated descriptor VLAD of the query image based on the reserved image local feature vector and the cluster center to which the reserved image local feature vector belongs; and normalize the VLAD of the query image to obtain a global feature vector of the query image.

[0181] In a possible implementation, the dimension reduction processing unit is further configured to perform dimension reduction processing on the global feature vector of the query image by using pre-generated third principal component analysis dimension reduction parameters.

[0182] In addition, the embodiment of the present disclosure further provides a device for generating a cluster center set. Referring to FIG. 5, it is a schematic diagram of a device for generating a cluster center set according to an embodiment of the present disclosure.

[0183] The device 500 comprises:

[0184] The third acquisition unit 501 is configured to acquire image local feature vectors of sample images of a target scene, wherein the sample images record dynamic targets and static targets.

[0185] The clustering unit 502 is configured to cluster the image local feature vectors to obtain cluster centers and store the cluster centers in a dictionary, wherein one cluster center corresponds to one index in the dictionary.

[0186] The fourth acquisition unit 503 is configured to acquire a mask of a dynamic target recorded in the sample images.

[0187] The second determination unit 504 is configured to determine cluster centers belonging to dynamic targets in the dictionary based on the mask of the dynamic target and mark the cluster centers in the dictionary.

[0188] In a possible implementation, the fourth acquisition unit 503 is specifically configured to input the sample image into a pre-trained image segmentation model to acquire segmentation masks corresponding to a plurality of key points in the sample image; and determine the mask of the dynamic target recorded in the sample image based on a category of the segmentation masks and a category of the dynamic target.

[0189] In a possible implementation, the second determination unit 504 is specifically configured to determine, for any key point in a sample image, a cluster center to which the key point belongs; determine a k-dimensional zero vector V based on a number k of cluster centers in the dictionary; update a value of an element Vi in the vector V based on a number of dynamic targets included in any cluster center i, wherein i = 1, 2, …, k; and determine, based on the values of the elements in the updated vector V, cluster centers corresponding to a preset number of elements as the cluster centers belonging to dynamic targets.

[0190] Based on the method embodiments and the device embodiments, the embodiments of the present disclosure further provide an electronic device. The following will be introduced with reference to the accompanying drawings.

[0191] Referring to FIG. 6, FIG. 6 is a schematic diagram of an electronic device provided by the embodiments of the present disclosure.

[0192] The device 600 includes a memory 601 and a processor 602.

[0193] The memory 601 is configured to store related program codes.

[0194] The processor 602 is configured to invoke the program codes and perform the method described in the above method embodiments.

[0195] In addition, the embodiments of the present disclosure further provide a computer readable storage medium, which is configured to store a computer program, and the computer program is configured to perform the method described in the above method embodiments.

[0196] In addition, the embodiments of the present disclosure further provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the method provided by the above method embodiments.

[0197] It should be noted that the technical features in the superordinate means provided in the embodiments of the present disclosure are clear to those skilled in the art, and the problems to be solved by the superordinate means are also clear. How to obtain the corresponding features can be selected by those skilled in the art according to the specific implementation requirements, and the means provided by the present disclosure should not be regarded as a limitation or the only implementation means.

[0198] It should be noted that the embodiments in the present specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts of each embodiment can be referred to each other. Especially, the system or device embodiments are described more simply because they are basically similar to the method embodiments. The relevant parts can be referred to the part of the description of the method embodiments. The device embodiments described above are only schematic, and the units or modules described as separate components can or can not be physically separated, and the components displayed as units or modules can or can not be physical modules, that is, they can be located in one place, or they can be distributed on multiple network units. According to the actual needs, some or all of the units or modules can be selected to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0199] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functions, and operations that can be implemented in methods, apparatus, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the block can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0200] It should be understood that, in the present disclosure, "at least one" means one or more, "multiple" means two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c, can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0201] It should also be noted that the relationship terms such as first and second in the present disclosure are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0202] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in Random Access Memory (RAM), flash memory, Read-Only Memory (ROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and

[0203] The above description of disclosed embodiments is intended to be illustrative and not restrictive. Many modifications of these embodiments by one having ordinary skill in the art are intended to be within the scope of the following claims. Thus, the disclosure is not to be limited to the examples presented but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A positioning method, wherein, The method includes: Obtain the local feature vector of the query image of the target scene, wherein the query image records the dynamic and static targets in the target scene; Based on the pre-generated set of cluster centers for the target scene, the cluster center to which the local feature vector of the image belongs and the index of the cluster center are obtained; From the image local feature vectors, delete the image local feature vectors of the dynamic target, where the index of the cluster center to which the image local feature vectors of the dynamic target belong belongs to the dynamic target; Based on the retained local feature vectors of the image and the cluster centers to which the retained local feature vectors belong, a global feature vector of the query image is generated; Based on the global feature vector of the query image, the location information corresponding to the sample image that matches the query image is obtained from the pre-generated image positioning fingerprint data; Based on the location information corresponding to the matched sample images, the location of the query image is determined.

2. The method according to claim 1, wherein, The process of obtaining the local feature vector of the query image of the target scene includes: The query image is input into a pre-trained feature extraction model to obtain the coordinates and feature vectors of the key points of the query image, as well as the coordinate regions and feature vectors of the pixel blocks of the query image. Based on the coordinates of the key points in the query image and the coordinate region of the pixel blocks in the query image, determine the pixel block to which the key points belong; The feature vectors of the key points in the query image are concatenated with the feature vectors of the pixel blocks to which the key points belong to obtain the local feature vectors of the image.

3. The method according to claim 2, wherein, The feature extraction model includes: a keypoint feature extraction model and a pixel block feature extraction model, wherein... The key point feature extraction model is used to output the coordinates and feature vectors of the key points of the query image; The pixel block feature extraction model is used to output the coordinate region and feature vector of the pixel block of the query image.

4. The method according to any one of claims 1-3, wherein, The process of obtaining the cluster center to which the local feature vector of the image belongs and the index of the cluster center based on the pre-generated cluster center set of the target scene includes: For each local feature vector of an image, obtain the distance from that local feature vector to each cluster center in the cluster center set; The cluster center with the shortest distance is taken as the cluster center to which the local feature vector of the image belongs, and the index of the cluster center to which it belongs is obtained from the set of cluster centers.

5. The method according to claim 2, wherein, Before concatenating the feature vectors of the key points in the query image with the feature vectors of the pixel blocks to which the key points belong, the method further includes: The feature vectors of key points in the query image are reduced in dimensionality using pre-generated first principal component analysis dimensionality reduction parameters; and / or, The feature vectors of the pixel blocks in the query image are reduced in dimensionality using pre-generated second principal component analysis dimensionality reduction parameters.

6. The method according to any one of claims 1-5, wherein, The process of generating a global feature vector for the query image based on the preserved local feature vectors of the image and the cluster centers to which the preserved local feature vectors belong includes: Based on the preserved local feature vectors of the image and the cluster centers to which the preserved local feature vectors of the image belong, a local clustering descriptor (VLAD) for the query image is generated; The local accumulation descriptor VLAD of the query image is normalized to obtain the global feature vector of the query image.

7. The method according to any one of claims 1-6, wherein, The method further includes: The global feature vector of the query image is reduced in dimensionality using pre-generated third principal component analysis dimensionality reduction parameters.

8. A method for generating a set of cluster centers, wherein, The method includes: Obtain the local feature vector of a sample image of the target scene, wherein the sample image records dynamic and static targets; Clustering is performed on the local feature vectors of the image to obtain cluster centers, which are stored in a dictionary. Each cluster center corresponds to an index in the dictionary. Obtain the mask of the dynamic target recorded in the sample image; Based on the mask of the dynamic target, the cluster centers belonging to the dynamic target in the dictionary are determined and marked in the dictionary.

9. The method according to claim 8, wherein, The process of obtaining the mask of the dynamic target recorded in the sample image includes: The sample image is input into a pre-trained image segmentation model to obtain the segmentation mask and its category corresponding to multiple key points in the sample image. Based on the category of the segmentation mask and the category of the dynamic target, the mask of the dynamic target recorded in the sample image is determined.

10. The method according to claim 9, wherein, Determining the cluster centers belonging to the dynamic target in the dictionary based on the mask of the dynamic target includes: For any key point in the sample image, determine the cluster center to which the key point belongs; Based on the number k of cluster centers in the dictionary, determine the k-dimensional zero vector V; For any cluster center i, update the value of element Vi in vector V based on the number of dynamic targets included in cluster center i, where i = 1, 2, ..., k; Based on the values ​​of each element in the updated vector V, a preset number of cluster centers are determined as the cluster centers belonging to the dynamic target.

11. An electronic device, wherein, Including memory and processor; The memory is used to store the relevant program code; The processor is used to call the program code to execute the method according to any one of claims 1-7 or to execute the method according to any one of claims 8-10.

12. A computer-readable storage medium, wherein, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1-7 or the method according to any one of claims 8-10.

13. A computer program product, wherein, Includes a computer program, which, when executed by a processor, implements the method as described in any one of claims 1-7 or performs the method as described in any one of claims 8-10.

Images