Image feature extraction method, target re-identification method, equipment and storage medium
By matching and fusing image feature extraction methods with computational cue pools, the problems of low computational efficiency and poor discrimination ability in traditional methods are solved, achieving more efficient image feature expression and vehicle recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-21
- Publication Date
- 2026-04-07
AI Technical Summary
Traditional image feature extraction methods suffer from low computational efficiency, waste of resources, and difficulty in distinguishing similar image content, especially in vehicle recognition where they perform poorly.
By acquiring the image to be processed and the computational cue pool, initial image features are extracted and matched with cue operators to determine the target cue operator and vector. Based on the computational parameters, computation and fusion are performed to obtain the final image features.
It improves the expressive power and computational efficiency of image features, enhances the performance of downstream tasks, and is particularly effective in distinguishing similar vehicles in vehicle re-identification.
Smart Images
Figure CN121811058A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, in particular to an image feature extraction method, a target re-identification method, equipment and a storage medium. BACKGROUND
[0002] Accurate and comprehensive image feature extraction of an image can ensure accurate execution of downstream tasks, such as target re-identification, image retrieval, image analysis and the like.
[0003] However, the traditional image feature extraction method relies on manual features (such as color histogram, texture feature), which has significant limitations; while the image feature extraction method based on deep learning has improved performance to some extent, but still faces two key problems: first, most models rely on fixed network structure, and it is difficult to flexibly adjust network parameters and computational complexity during training and inference, resulting in low computational efficiency of feature extraction and resource waste or performance degradation; it is difficult to extract discriminative basis for distinguishing similar image contents, such as vehicles as standard industrial products, which have similar appearance features, making it difficult to extract image features for distinguishing similar vehicles, resulting in poor vehicle recognition effect.
[0004] Therefore, how to extract accurate and available image features is a problem to be solved by those skilled in the art. SUMMARY
[0005] In order to solve the above technical problems, the present application at least provides an image feature extraction method, a target re-identification method, equipment and a storage medium.
[0006] The first aspect of the present application provides an image feature extraction method, the method comprising: acquiring a to-be-processed image and an operation prompt pool, the operation prompt pool containing a prompt vector and a prompt operator, the prompt operator containing operation parameters for operating on the prompt vector and / or input image features; extracting image features of the to-be-processed image to obtain initial image features, matching the initial image features with the prompt operator in the operation prompt pool to obtain a target prompt operator; determining the prompt vector matched with the target prompt operator and / or the initial image features to obtain matching features; operating on the matching features based on the operation parameters of the target prompt operator to obtain operation result features; and fusing the initial image features and the operation result features to obtain final image features corresponding to the to-be-processed image.
[0007] In an embodiment, before matching the initial image feature and the prompt operator in the operation prompt pool to obtain the target prompt operator, further comprising: inputting the initial image feature into a pre-trained gating function, the gating function determining whether to directly fuse the initial image feature and the prompt vector or use the prompt operator for operation; if the output result of the gating function indicates that the initial image feature and the prompt vector are directly fused, matching the initial image feature and the prompt vector in the operation prompt pool to obtain the target prompt vector, fusing the initial image feature and the target prompt vector to obtain the operation result feature; if the output result of the gating function indicates that the prompt operator is used for operation, matching the initial image feature and the prompt operator in the operation prompt pool to obtain the target prompt operator, and calculating the operation result feature based on the target prompt operator.
[0008] In an embodiment, matching the initial image feature and the prompt vector in the operation prompt pool to obtain the target prompt vector comprises: calculating the similarity between the initial image feature and each prompt vector to obtain a vector similarity; selecting a prompt vector whose vector similarity satisfies a preset first selection condition to obtain the target prompt vector.
[0009] In an embodiment, each prompt operator is respectively provided with an index vector; matching the initial image feature and the prompt operator in the operation prompt pool to obtain the target prompt operator comprises: calculating the similarity between the initial image feature and the index vector of each prompt operator to obtain an operator similarity; selecting a prompt operator whose operator similarity satisfies a preset second selection condition to obtain the target prompt operator.
[0010] In an embodiment, determining the matching feature of the target prompt operator, the prompt vector and / or the initial image feature comprises: determining the input data type corresponding to the target prompt operator; if the input data type includes an image feature, taking the initial image feature as the matching feature; if the input data type includes a prompt vector, determining a prompt vector matched with the initial image feature to obtain the matching feature; if the input data type includes a prompt vector and an image feature, taking the initial image feature and the prompt vector matched with the initial image feature as the matching feature.
[0011] In an embodiment, the operation prompt pool further contains local image features; the method further comprises: segmenting the to-be-processed image to obtain an image block, and extracting image features of the image block to obtain local image features; calculating similarities between the local image features corresponding to the to-be-processed image and the local image features in the operation prompt pool to obtain image similarities; selecting, from the operation prompt pool, local image features whose image similarities satisfy a preset third selection condition to obtain target local features; and fusing the initial image features and the operation result features to obtain final image features corresponding to the to-be-processed image, including: calculating a weight parameter corresponding to the target local features based on the image similarity corresponding to the target local features; and fusing the target local features, the initial image features and the operation result features based on the weight parameter to obtain the final image features corresponding to the to-be-processed image.
[0012] In an embodiment, the operation prompt pool further contains local image features; the method further comprises: obtaining frequencies at which each prompt vector and / or prompt operator and / or local image feature in the operation prompt pool is selected in the model training stage; and deleting prompt vectors and / or prompt operators and / or local image features in the operation prompt pool whose frequencies satisfy a preset deletion condition.
[0013] The second aspect of the present application provides a target re-identification method, which comprises: obtaining a to-be-processed image, performing target detection on the to-be-processed image to obtain a target image corresponding to the to-be-processed image; performing image feature extraction on the target image by using an image feature extraction method to obtain final image features corresponding to the target image; and performing target re-identification based on the final image features.
[0014] The third aspect of the present application provides an image feature extraction device, which comprises: a data acquisition module configured to acquire a to-be-processed image and an operation prompt pool, the operation prompt pool containing prompt vectors and prompt operators, the prompt operators containing operation parameters for performing operations on the prompt vectors and / or input image features; an operator matching module configured to extract image features of the to-be-processed image to obtain initial image features, and match the initial image features and the prompt operators in the operation prompt pool to obtain target prompt operators; a feature matching module configured to determine prompt vectors and / or initial image features matched by the target prompt operators to obtain matching features; an operation module configured to perform operations on the matching features based on the operation parameters of the target prompt operators to obtain operation result features; and a fusion module configured to fuse the initial image features and the operation result features to obtain final image features corresponding to the to-be-processed image.
[0015] The fourth aspect of the present application provides a target re-identification device, the device comprising: a target detection module, configured to obtain a to-be-processed image, perform target detection on the to-be-processed image, and obtain a target image corresponding to the to-be-processed image; a feature extraction module, configured to perform image feature extraction on the target image by using an image feature extraction method, and obtain final image features corresponding to the target image; and a re-identification module, configured to perform target re-identification based on the final image features.
[0016] The fifth aspect of the present application provides an electronic device comprising a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the image feature extraction method or the target re-identification method.
[0017] The sixth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, the program instructions being executed by a processor to implement the image feature extraction method or the target re-identification method.
[0018] The above scheme extracts image features of a to-be-processed image to obtain initial image features, matches the initial image features with prompt operators in a prompt pool to obtain a target prompt operator, determines a prompt vector matched with the target prompt operator and / or the initial image features to obtain matching features, performs operation on the matching features based on operation parameters of the target prompt operator to obtain operation result features, and fuses the initial image features and the operation result features to obtain final image features corresponding to the to-be-processed image. The prompt operator can be used to flexibly operate the prompt vector and / or the initial image features, and can adaptively operate the prompt vector or the initial image features according to actual conditions, thereby improving the flexibility of information supplement, reducing the collection cost of the prompt vector, possibly capturing information that cannot be expressed by the prompt vector or the initial image features alone, improving the utilization rate of the prompt vector, enhancing the feature expression capability of the operation result features, and further enhancing the feature expression capability of the final image features, thereby facilitating downstream task execution.
[0019] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than limiting the present application. BRIEF DESCRIPTION OF DRAWINGS
[0020] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the technical solutions of the present application.
[0021] Figure 1 is a schematic diagram of a scheme implementation environment illustrated by an exemplary embodiment of the present application; Figure 2 is a flowchart of an image feature extraction method illustrated by an exemplary embodiment of the present application; Figure 3is a schematic diagram of image feature extraction according to an example embodiment of the present application; Figure 4 is a flow chart of a target re-identification method according to an example embodiment of the present application; Figure 5 is a block diagram of an image feature extraction apparatus according to an example embodiment of the present application; Figure 6 is a block diagram of a target re-identification apparatus according to an example embodiment of the present application; Figure 7 is a structural schematic diagram of an electronic device according to an example embodiment of the present application; Figure 8 is a structural schematic diagram of a computer readable storage medium according to an example embodiment of the present application. DETAILED DESCRIPTION
[0022] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0023] In the following description, specific details such as specific system structures, interfaces, techniques, etc. are presented in order to provide a thorough understanding of the present application for the sake of explanation, rather than for the sake of limitation.
[0024] The term "and / or" herein is merely a description of the association information of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.
[0025] The image feature extraction method provided by the embodiments of the present application will be described below.
[0026] Please refer to Figure 1 , Figure 1 is a schematic diagram of a scheme implementation environment according to an example embodiment of the present application. The scheme implementation environment can include a terminal 110 and a server 120, which are connected to each other in communication.
[0027] The number of terminals 110 can be one or more. The terminal 110 can be a camera, a smartphone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.
[0028] The server 120 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.
[0029] In one example, the server 120 can perform image feature extraction on the to-be-processed image obtained from the terminal 110 to obtain the final image feature. Of course, the server 120 can store the final image feature locally, return it to the terminal 110, or transmit it to another terminal.
[0030] In one example, the terminal 110 has a client installed and running a target application, which can be an application that provides image feature extraction functions. The target application is used to extract the final image feature corresponding to the to-be-processed image. The server 120 can be a background server of the target application, which provides background services for the client of the target application.
[0031] The image feature extraction method provided by the embodiments of the present application can be executed by the terminal 110, such as the client of the target application installed and running in the terminal 110, or by the server 120, or by the terminal 110 and the server 120 in cooperation, that is, part of the steps of the method are executed by the terminal 110 and the other part of the steps are executed by the server 120.
[0032] Referring to Figure 2 , Figure 2 is a flowchart of an image feature extraction method according to an example embodiment of the present application. The image feature extraction method can be applied to the implementation environment shown in Figure 1 and executed by the server in the implementation environment. It should be understood that the method can also be applied to other example implementation environments and executed by devices in other implementation environments, and the present embodiment does not limit the implementation environment to which the method is applied.
[0033] As shown in Figure 2 , the image feature extraction method includes at least steps S210 to S250, which are described in detail as follows: Step S210: Obtain a to-be-processed image and an operation hint pool, the operation hint pool containing a hint vector and a hint operator, the hint operator containing operation parameters for operating on the hint vector and / or input image features.
[0034] The operation hint pool at least contains the hint vector and the hint operator.
[0035] For example, each prompt vector pi is a d-dimensional vector, where i∈{1, 2, …, M}, M is the number of prompt vectors, and the prompt vector is used to represent specific semantic or task-related information.
[0036] For example, the generation of the prompt vector includes: Random initialization: sampling from a preset Gaussian distribution or a preset uniform distribution, suitable for scenarios without prior knowledge; Prior knowledge encoding: encoding the prior knowledge to obtain the prompt vector, the prior knowledge includes but is not limited to one or more of text, image, and audio data, for example, obtaining prior task data, and encoding the prior task data to obtain the prompt vector, wherein a pre-trained language model (such as a pre-trained language model based on the Transformer architecture, such as the pre-trained language model BERT (Bidirectional Encoder Representations from Transformers), the generative pre-trained (GPT) model, etc.) or a video model (such as the contrastive language-image pre-training (CLIP) model) can be used to encode the prior knowledge; Original model extraction: obtaining the feature vector corresponding to the network in the task-related neural network model to initialize the prompt vector, for example, directly using the feature vector corresponding to a certain layer of the training network before the operation prompt pool is added (at the position where the operation prompt pool is to be added) to initialize the prompt vector.
[0037] Of course, other ways of generating prompt vectors can also be used, which are not limited by the present application.
[0038] The generated prompt vector is stored in the form of a matrix P∈ , which facilitates batch calculation and retrieval.
[0039] For example, each prompt operator Tj is a learnable transformation function, where j∈{1, 2, …, N}, N is the number of prompt operators, and the prompt operator contains operation parameters for operating on the prompt vector and / or the input image features, for example, the prompt operator is used to dynamically adjust or combine the prompt vector.
[0040] For example, the types of prompt operators include one or more of linear transformation operators, nonlinear activation operators, attention weighting operators, and combination operators.
[0041] The linear transformation operator is used for linear transformation of the input features. For example, the linear transformation operator is a linear operation layer, but realizes transformation of the input feature dimension.
[0042] The nonlinear activation operator is used for nonlinear activation of the input features. For example, the nonlinear activation operator includes but is not limited to a rectified linear unit (ReLU), a Gaussian error linear unit (GeLU), etc., for enhancing the expression ability of the input features.
[0043] The attention weighting operator is used for weighted calculation of the input features. For example, the attention weighting operator dynamically calculates weights based on the initial image features, and performs weighted fusion of multiple input features according to the weights.
[0044] The combination operator is used for fusion of the input features. For example, the combination operator is used for concatenation, additive fusion, etc. of multiple input features.
[0045] The input features mentioned above refer to a feature vector input into the prompt operator for operation.
[0046] It should be noted that the operation parameters in the prompt operator are all learnable, which are optimized and learned through back propagation in the training process. The specific training methods include but are not limited to supervised training, semi-supervised training, unsupervised training, etc., and are determined according to the required downstream task. The present application does not limit this.
[0047] Step S220: extracting image features of the to-be-processed image to obtain initial image features, matching the initial image features with the prompt operators in the operation prompt pool to obtain a target prompt operator.
[0048] The image features of the to-be-processed image are extracted, and the extracted image features are taken as initial image features.
[0049] The extracted image features can include hand-crafted features (such as color features, texture features, shape features, etc.) and / or deep features. A pre-trained feature extraction neural network can be used for image feature extraction of the to-be-processed image. The feature extraction neural network can be implemented based on a convolutional neural network (CNN), a recurrent neural network (RNN), etc., and the present application does not limit this.
[0050] The initial image features are matched with the prompt operators in the operation prompt pool to obtain a target prompt operator.
[0051] Optionally, to facilitate the selection of appropriate cue operators for the initial image features during model inference, each cue operator is equipped with a key, and each key is a vector. The cue operator to be used is selected by calculating the similarity between the key and the input initial image features.
[0052] During model training, the initial value of the key for the cue operator can be randomly initialized, or the feature vector of the sample image in the training samples can be used as the initial value of the key. If a cue operator is not selected for a long time, the value of the key is updated again, such as by replacing it with the feature vector of the sample image input in the next iteration. In subsequent model inference, the cue operator suitable for the current initial image features can be determined by calculating the similarity between the key and the cue operator.
[0053] Optionally, in addition to setting a key for each cue operator to achieve subsequent cue operator matching, different types of images can be pre-classified, and corresponding cue operators can be trained independently for different types of images. During the model inference process, the matching cue operator is determined according to the image type to which the image to be processed belongs.
[0054] Optionally, the matching cue operator can be determined based on the image quality of the image to be processed. Image quality can be determined based on image sharpness, image brightness, target occlusion, target pose, etc. For example, taking a vehicle re-identification scenario as an example, if the vehicle's pose is the first pose and the first key part of the vehicle is visible, then the image quality is higher than the threshold, and cue operator T1 is selected as the target cue operator; if the vehicle's pose is the first pose and the first key part of the vehicle is not visible, then the image quality is lower than the threshold, and cue operators T1 and T2 are selected as the target cue operators.
[0055] Step S230: Determine the cue vector and / or initial image features for target cue operator matching to obtain matching features.
[0056] After obtaining the target cue operator, determine the cue vector and / or initial image features that need to participate in the operation to obtain the matching features.
[0057] For example, similarity calculations are performed on the initial image features and the cue vectors to select matching cue vectors and obtain matching features.
[0058] For example, the cue operator may pre-specify the features to be used, and the matching features are obtained based on the features pre-specified by the target cue operator. For instance, if the target cue operator pre-specifies the cue vectors to be used in the operation as the initial input image features, then the initial image features are used as the matching features; if the target cue operator pre-specifies the cue vectors to be used in the operation as cue vector p1 and cue vector p2, then cue vector p1 and cue vector p2 are used as the matching features.
[0059] Step S240: Perform operations on the matching features based on the operation parameters of the target cue operator to obtain the operation result features.
[0060] Based on the operation parameters of the target prompt operator, the matching features are processed to obtain the result features.
[0061] For example, linear transformations and / or nonlinear activations can be applied to matching features, and / or attention weighting, concatenation, and / or element-wise addition can be performed on multiple matching features to obtain the resulting features.
[0062] Step S250: Fuse the initial image features and the operation result features to obtain the final image features corresponding to the image to be processed.
[0063] The initial image features and the processed features can be directly concatenated or added to achieve fusion and obtain the final image features; alternatively, the fusion weights corresponding to the initial image features and the processed features can be obtained and weighted fusion can be performed to obtain the final image features.
[0064] For example, there may be pre-set fusion rules between initial image features and computation result features, meaning that the fusion rules are the same for each image to be processed.
[0065] For example, the fusion rules can be determined based on the specific circumstances, and the initial image features and the operation result features can be fused based on the fusion rules to obtain the final image features.
[0066] For example, the image quality of the image to be processed is determined, and the fusion weights corresponding to the initial image features and the processed result features are determined based on the image quality. The lower the image quality, the higher the fusion weight of the processed result features. Alternatively, the fusion rule is determined based on the type of the target cue operator. If the target cue operator is a linear transformation operator, the first fusion rule is used; if the target cue operator is an attention-weighted operator, the second fusion rule is used. Or, the target cue operator also includes a fusion rule, which can be obtained during model training. The initial image features and the processed result features are fused based on the fusion rule corresponding to the target cue operator to obtain the final image features.
[0067] The image feature extraction method provided in this application obtains initial image features by extracting image features from the image to be processed. The initial image features are then matched with cue operators in a cue pool to obtain a target cue operator. The cue vector and / or initial image features matched with the target cue operator are determined to obtain matching features. The matching features are then processed based on the operational parameters of the target cue operator to obtain the operational result features. Finally, the initial image features and the operational result features are fused to obtain the final image features corresponding to the image to be processed. The cue operator allows for flexible computation on the cue vector and / or initial image features, enabling adaptive computation based on actual conditions. This improves the flexibility of information supplementation, reduces the cost of collecting cue vectors, and may capture information that cue vectors or initial image features cannot express alone, improving the utilization rate of cue vectors and enhancing the feature representation capability of the operational result features. This, in turn, enhances the feature representation capability of the final image features, facilitating downstream task execution.
[0068] The following describes some embodiments of this application in detail.
[0069] In some implementations, steps S301 to S304 are included before matching the initial image features with the cue operators in the cue pool to obtain the target cue operator.
[0070] Step S301: Input the initial image features into a pre-trained gating function. The gating function determines whether the initial image features and the cue vector are directly fused or operated using a cue operator.
[0071] The gating function can be obtained through pre-training; or it can be preset based on experience. This application does not impose any restrictions on this.
[0072] For example, the gating function can determine the image quality of the image to be processed. If the image quality is higher than a threshold, the initial image features and the cue vector are directly fused to save computing resources. If the image quality is not higher than the threshold, the cue operator is used to ensure that the final image features can meet the requirements of downstream tasks.
[0073] Of course, in addition to using image quality as an example as described above, other methods can be used for judgment, and this application does not limit this.
[0074] Step S302: If the output result of the gating function indicates that the initial image features and the cue vector are directly fused, then proceed to step S303; if the output result of the gating function indicates that the cue operator is used for the operation, then proceed to step S304.
[0075] Step S303: Match the initial image features with the cue vectors in the cue pool to obtain the target cue vector, and fuse the initial image features and the target cue vector to obtain the operation result features.
[0076] Specifically, the similarity between the initial image features and each cue vector is calculated to obtain the vector similarity; the cue vector whose vector similarity satisfies the preset first selection condition is selected to obtain the target cue vector.
[0077] The vector similarity can be cosine similarity, or similarity obtained based on the initial image features and the Euclidean distance or Manhattan distance between each cue vector; this application does not limit this.
[0078] For example, select suggestion vectors whose vector similarity is greater than a first threshold. The first threshold can be preset based on experience or obtained during model training.
[0079] For example, select the m most similar cue vectors. Here, m can be preset based on experience or obtained during model training.
[0080] Then, the initial image features and the target cue vector are fused to obtain the computational result features.
[0081] Similarly, the initial image features and the target cue vector can be directly concatenated or added to achieve fusion and obtain the resulting features; alternatively, the fusion weights corresponding to the initial image features and the target cue vector can be obtained and weighted fusion can be performed to obtain the resulting features.
[0082] For example, the similarity of the vectors corresponding to each target cue vector is normalized to obtain the weights of each target cue vector. Based on the weights of each target cue vector, the initial image features and each target cue vector are weighted and fused to obtain the resulting features.
[0083] Step S304: Match the initial image features with the cue operators in the cue pool to obtain the target cue operator, and calculate the operation result features based on the target cue operator.
[0084] In some implementations, each cue operator is provided with an index vector; in step S220, the initial image features are matched with the cue operators in the cue pool to obtain the target cue operator, including steps S221 to S222.
[0085] Step S221: Calculate the similarity between the initial image features and the index vectors of each cue operator to obtain the operator similarity.
[0086] The calculation method for operator similarity is similar to that for vector similarity, and will not be elaborated here.
[0087] Step S222: Select the prompt operator whose operator similarity satisfies the preset second selection condition to obtain the target prompt operator.
[0088] For example, select prompt operators whose operator similarity is greater than a second threshold. The second threshold can be preset based on experience or obtained during model training.
[0089] For example, select the n most similar prompt operators. Here, n can be preset based on experience or obtained during model training.
[0090] Then, the cue vector and / or initial image features for target cue operator matching are determined to obtain matching features.
[0091] In some implementations, step S230 involves determining the cue vector and / or initial image features matched by the target cue operator to obtain matching features, including steps S231 to S235 below.
[0092] Step S231: Determine the input data type corresponding to the target prompt operator.
[0093] Step S232: If the input data type includes image features, then proceed to step S233; if the input data type includes cue vectors, then proceed to step S234; if the input data type includes both cue vectors and image features, then proceed to step S235.
[0094] Step S233: Use the initial image features as matching features.
[0095] Step S234: Determine the cue vector that matches the initial image features to obtain the matching features.
[0096] The matching method between the initial image features and the cue vector is described in step S303 above and will not be repeated here.
[0097] Step S235: Use the initial image features and the cue vectors that match the initial image features as matching features.
[0098] The matching features are used as input features to the target cue operator. The resulting features are then processed to obtain the final image features corresponding to the image to be processed.
[0099] In some implementations, the computational cue pool also contains local image features; the method further includes the following steps S401 to S403.
[0100] Step S401: Segment the image to be processed to obtain image blocks, and extract the image features of the image blocks to obtain local image features.
[0101] Image blocks can be obtained by segmenting the image based on the type of each pixel; or the image to be processed can be segmented proportionally or unequally according to preset segmentation parameters to obtain image blocks. This application does not limit the specific method used.
[0102] Taking vehicle re-identification as an example, the image to be processed is a vehicle image. The vehicle image is segmented using a pre-trained vehicle local segmentation model to obtain vehicle local image blocks (such as door image blocks, license plate image blocks, front window image blocks, etc.). Then, each vehicle local image block is scaled to the same size as the complete vehicle image, and then image features are extracted by an image encoder to obtain local image features.
[0103] Step S402: Calculate the similarity between the local image features corresponding to the image to be processed and the local image features in the operation prompt pool to obtain the image similarity.
[0104] To improve the accuracy of local image feature matching, the calculation method of local image features in the computational cue pool is the same as that of local image features of the image to be processed, that is, the same image block segmentation network and image feature extraction network are used.
[0105] Specifically, a set of sample images is obtained, and the sample images in the set are segmented and their image features are extracted in sequence to obtain local image features. The local image features corresponding to the sample images are then added to the computational cue pool.
[0106] Step S403: Select local image features whose image similarity satisfies the preset third selection condition from the operation prompt pool to obtain the target local features.
[0107] For example, select cue vectors whose image similarity is greater than a third threshold. This third threshold can be pre-set based on experience or obtained during model training.
[0108] For example, select the q cue vectors with the highest image similarity. Here, q can be preset based on experience or obtained during model training.
[0109] Then, in step S250, the initial image features and the operation result features are fused to obtain the final image features corresponding to the image to be processed, including: fusing the initial image features, target local features and operation result features to obtain the final image features corresponding to the image to be processed.
[0110] For example, based on the image similarity corresponding to the target local features, the weight parameters corresponding to the target local features are calculated; based on the weight parameters, the target local features, the initial image features, and the operation result features are fused to obtain the final image features corresponding to the image to be processed.
[0111] For example, based on the image similarity corresponding to the target local feature, the reciprocal of the image similarity is calculated. Then, the reciprocals of the image similarity of all target local features are normalized to obtain the weight parameter corresponding to each target local feature. Then, based on the weight parameter corresponding to each target local feature, the target local feature, the initial image feature, and the operation result feature are weighted and fused to obtain the final image feature.
[0112] In this method, normalizing the inverse of the image similarity enhances the influence of local features that differ significantly from the image to be processed in each target local feature, thereby encouraging the model to focus on these local features.
[0113] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating image feature extraction in an exemplary embodiment of this application, such as... Figure 3 As shown, the number of cue vectors in the cue pool is M, the number of cue operators is N, and the number of local image features is Q. The initial image features of the image to be processed and the cue operators and / or cue vectors are calculated to obtain the operation result features. The local image features of the image to be processed and the local image features in the cue pool are calculated to obtain the matching target local features. Then, the initial image features, the operation result features, and the target local features are fused to obtain the final image features.
[0114] In some implementations, the computational cue pool contains cue vectors, cue operators, and local image features; the method further includes: obtaining the frequency at which each cue vector and / or cue operator and / or local image feature is selected in the computational cue pool during the model training phase; and deleting cue vectors and / or cue operators and / or local image features in the computational cue pool whose frequencies meet preset deletion conditions.
[0115] During model training, the frequency of selection for each cue vector, cue operator, and local image feature is recorded. During model inference, based on the frequency of selection for each cue vector, cue operator, and local image feature, some low-frequency cue vectors, cue operators, and local image features are deleted to obtain the computational cue pool used in model inference.
[0116] For example, sort each cue vector in descending order of frequency, and then delete the bottom 50% of the boosting vectors; sort each cue operator in descending order of frequency, and then delete the bottom 50% of the cue operators; sort each local image feature in descending order of frequency, and then delete the bottom 80% of the local image features.
[0117] Of course, a frequency threshold can also be set to delete cue vectors, cue operators, and local image features whose frequencies are lower than the frequency threshold. This application does not limit this.
[0118] It should be noted that the deletion ratio, frequency threshold, etc. mentioned above can be set based on experience, or can be flexibly calculated according to the actual application scenario. For example, the deletion ratio or frequency threshold can be determined based on the amount of available computing resources of the model deployment equipment.
[0119] The above embodiments can improve model performance by prioritizing the retention of superior cue vectors, cue operators, and local image features obtained during training, while reducing the number of model parameters in the inference process.
[0120] In some implementations, the number of cue vectors, and / or cue operators, and / or local image features selected can be determined based on the amount of available computing resources and / or the image quality of the image to be processed. For example, if the image quality is lower and more computing resources are available, more cue vectors, and / or cue operators, and / or local image features are selected to improve the quality of the final image features; if the image quality is higher and fewer computing resources are available, more cue vectors, and / or cue operators, and / or local image features are selected to avoid excessive device load.
[0121] The extracted final image features are used to realize downstream tasks, which can be any task that requires the use of image features, such as target re-identification, image retrieval, or image analysis. This application does not limit this.
[0122] Let's take target re-identification as an example to illustrate this.
[0123] For example, please see Figure 4 , Figure 4 This is a flowchart illustrating a target re-identification method in an exemplary embodiment of this application, as shown below. Figure 4 As shown, it includes the following steps S410 to S430.
[0124] Step S410: Obtain the image to be processed, perform target detection on the image to be processed, and obtain the target image corresponding to the image to be processed.
[0125] The target of object detection is the target that needs to be re-identified, including but not limited to vehicles, pedestrians, and animals.
[0126] Step S420: Use image feature extraction methods to extract image features from the target image to obtain the final image features corresponding to the target image.
[0127] The image feature extraction methods described in the above embodiments are used to extract image features from the target image to obtain the final image features.
[0128] Step S430: Perform target re-identification based on the final image features.
[0129] Based on the final image features, target re-identification is performed.
[0130] For example, if the image to be processed is the current image frame, the final image features corresponding to each target extracted from the previous image frames acquired before the current image frame are obtained to obtain the features to be matched. The similarity between the current final image features and the features to be matched is calculated. If there is a target in the previous image frame whose final image features have a similarity greater than a preset threshold with the current final image features, it is determined that the target re-identification is successful, and the identifier of the target in the previous image frame is copied to the target corresponding to the current final image features; if not, a new identifier is directly generated for the target corresponding to the current final image features.
[0131] The above embodiments can improve the expressive power of the final image features and improve the accuracy of target re-identification. Especially for vehicle re-identification tasks, even if the basis for distinguishing different vehicles may only occupy a small part of the vehicle (such as annual inspection stickers, items placed on the front of the vehicle, etc.), the feature expression power can be enhanced by the computational prompt pool, which can effectively distinguish similar vehicles and improve the re-identification effect.
[0132] Figure 5 This is a block diagram illustrating an image feature extraction apparatus according to an exemplary embodiment of this application. Figure 5 As shown, the exemplary image feature extraction device 500 includes: The data acquisition module 510 is used to acquire the image to be processed and the operation prompt pool. The operation prompt pool contains prompt vectors and prompt operators. The prompt operators contain operation parameters for performing operations on the prompt vectors and / or the input image features. The operator matching module 520 is used to extract image features of the image to be processed to obtain initial image features, and match the initial image features with the prompt operators in the operation prompt pool to obtain the target prompt operator; The feature matching module 530 is used to determine the cue vector and / or initial image features for matching the target cue operator, and to obtain the matching features; The operation module 540 is used to perform operations on the matching features based on the operation parameters of the target prompting operator to obtain the operation result features; The fusion module 550 is used to fuse the initial image features and the operation result features to obtain the final image features corresponding to the image to be processed.
[0133] Figure 6 This is a block diagram illustrating a target recognition device in an exemplary embodiment of this application. Figure 6 As shown, the exemplary target re-identification device 600 includes: The target detection module 610 is used to acquire the image to be processed, perform target detection on the image to be processed, and obtain the target image corresponding to the image to be processed. The feature extraction module 620 is used to extract image features from the target image using an image feature extraction method to obtain the final image features corresponding to the target image. The re-identification module 630 is used to perform target re-identification based on the final image features.
[0134] It should be noted that the image feature extraction device and target re-identification device provided in the above embodiments belong to the same concept as the image feature extraction method and target re-identification method provided in the above embodiments. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image feature extraction device and target re-identification device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.
[0135] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment of this application. The electronic device 700 includes a memory 701 and a processor 702. The processor 702 executes program instructions stored in the memory 701 to implement the steps in any of the above-described image feature extraction method embodiments. In a specific implementation scenario, the electronic device 700 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 700 may also include mobile devices such as laptops and tablets, without limitation.
[0136] Specifically, processor 702 controls itself and memory 701 to implement the steps in any of the above-described image feature extraction methods or target re-identification method embodiments. Processor 702 can also be referred to as a Central Processing Unit (CPU). Processor 702 may be an integrated circuit chip with signal processing capabilities. Processor 702 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 702 can be implemented using integrated circuit chips.
[0137] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating the structure of a computer-readable storage medium in an exemplary embodiment of this application. The computer-readable storage medium 800 stores program instructions 810 that can be executed by a processor. The program instructions 810 are used to implement the steps in any of the above-described image feature extraction methods or target re-identification method embodiments.
[0138] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0139] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0140] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.
[0141] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. An image feature extraction method, characterized in that, The method includes: Obtain the image to be processed and the computational cue pool, wherein the computational cue pool contains cue vectors and cue operators, and the cue operators contain computational parameters for performing operations on the cue vectors and / or the input image features; Extract image features from the image to be processed to obtain initial image features, and match the initial image features with the prompt operators in the operation prompt pool to obtain the target prompt operator; Determine the cue vector matched by the target cue operator and / or the initial image features to obtain the matching features; The matching features are calculated based on the operational parameters of the target prompting operator to obtain the operational result features; The initial image features and the computation result features are fused to obtain the final image features corresponding to the image to be processed.
2. The method according to claim 1, characterized in that, Before matching the initial image features with the cue operators in the cue pool to obtain the target cue operator, the method further includes: The initial image features are input into a pre-trained gating function, which determines whether the initial image features and the cue vector are directly fused or processed using a cue operator. If the output result of the gating function indicates that the initial image features and the cue vector are directly fused, then the initial image features and the cue vectors in the computational cue pool are matched to obtain the target cue vector, and the initial image features and the target cue vector are fused to obtain the computational result features; If the output result of the gate function indicates that the operation is performed using a cue operator, then the initial image features are matched with the cue operators in the operation cue pool to obtain the target cue operator, and the operation result features are calculated based on the target cue operator.
3. The method according to claim 2, characterized in that, The step of matching the initial image features with the cue vectors in the computational cue pool to obtain the target cue vector includes: Calculate the similarity between the initial image features and each cue vector to obtain the vector similarity. Select the prompt vector whose vector similarity satisfies the preset first selection condition to obtain the target prompt vector.
4. The method according to claim 1, characterized in that, Each cue operator is assigned an index vector; the process of matching the initial image features with the cue operators in the cue pool to obtain the target cue operator includes: Calculate the similarity between the initial image features and the index vectors of each cue operator to obtain the operator similarity; Select the prompt operator whose operator similarity satisfies the preset second selection condition to obtain the target prompt operator.
5. The method according to claim 1, characterized in that, The step of determining the cue vector matched by the target cue operator and / or the initial image features to obtain matching features includes: Determine the input data type corresponding to the target prompting operator; If the input data type includes image features, then the initial image features are used as matching features; If the input data type includes a cue vector, then the cue vector that matches the initial image features is determined, and the matching features are obtained; If the input data type includes cue vectors and image features, then the initial image features and the cue vectors that match the initial image features are used as matching features.
6. The method according to any one of claims 1 to 5, characterized in that, The computational prompt pool also contains local image features; the method further includes: The image to be processed is segmented to obtain image blocks, and the image features of the image blocks are extracted to obtain local image features; Calculate the similarity between the local image features corresponding to the image to be processed and the local image features in the operation prompt pool to obtain the image similarity; Local image features whose image similarity satisfies the preset third selection condition are selected from the computational prompt pool to obtain the target local features; The step of fusing the initial image features and the computation result features to obtain the final image features corresponding to the image to be processed includes: Based on the image similarity corresponding to the target local features, calculate the weight parameters corresponding to the target local features; Based on the weight parameters, the target local features, the initial image features, and the operation result features are fused to obtain the final image features corresponding to the image to be processed.
7. The method according to any one of claims 1 to 5, characterized in that, The computational prompt pool also contains local image features; the method further includes: Obtain the frequency at which each cue vector and / or cue operator and / or local image feature is selected in the computational cue pool during the model training phase; Delete the prompt vectors and / or prompt operators and / or local image features in the operation prompt pool whose frequency meets the preset deletion conditions.
8. A target re-identification method, characterized in that, The method includes: Obtain the image to be processed, perform target detection on the image to be processed, and obtain the target image corresponding to the image to be processed; The image feature extraction method according to any one of claims 1 to 7 is used to extract image features from the target image to obtain the final image features corresponding to the target image; Target re-identification is performed based on the final image features.
9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the image feature extraction method as described in any one of claims 1-7, or the target re-identification method as described in claim 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the image feature extraction method as described in any one of claims 1-7 or the target re-identification method as described in claim 8.