A human body image re-identification method and device, computer equipment and a storage medium

Abstract

The present application relates to the technical field of image processing, and particularly relates to a human body image re-identification method and device, computer equipment and a storage medium. By acquiring N pieces of to-be-identified human body images and M pieces of database images stored in a database, the image features of the to-be-identified human body images and the database images are fused, the fused features are enhanced through a self-attention mechanism, the target images matched with the to-be-identified human body images are identified according to the enhanced fused features, and a re-identification result is obtained. In the present application, the features of the human body images in the database are used to enhance the features of the to-be-identified images, and the enhanced features are used for identification in the database. Since the features of the to-be-identified images are enhanced by using the features of the human body images in the database, the correlation between the enhanced features and the features of the human body images in the database is improved, so that the corresponding re-identification rate can be improved when the human body images are re-identified.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, computer device, and storage medium for human image re-identification. Background Technology

[0002] In recent years, with the increasing demand in the fields of intelligent security and video surveillance, research on human image re-identification has received more and more attention and research. In video surveillance, due to factors such as low camera resolution, insufficient light intensity, poor camera angle, and object obstruction, it is difficult to capture clear facial information of human bodies, making it difficult to identify human identities through facial information. Human image re-identification can be seen as an image recognition task, using computer vision technology to determine whether a specific human body exists in a given database of videos and images. This technology can avoid the situation where it is difficult to identify human identities through facial information due to factors such as low camera resolution, insufficient light intensity, poor camera angle, and object obstruction.

[0003] Currently, the process of human image re-identification can be broadly divided into two stages. The first stage involves using a feature extractor to extract features from the image of the human to be identified and images in a human database. The second stage involves finding images from the database that match the human to be identified based on the feature similarity between the two images. However, when using a feature extractor to extract image features, the extracted features have low relevance within the same file, resulting in a low re-identification rate for human images. Therefore, improving the re-identification rate of human images has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method, apparatus, computer device, and storage medium for re-identifying human images to solve the problem of low re-identification rate of human images.

[0005] In a first aspect, embodiments of the present invention provide a human image re-identification method, the human image re-identification method comprising:

[0006] Obtain N human images to be identified, and M database images stored in the database, where N and M are both integers greater than zero;

[0007] Extract image features from the N images of the human body to be identified to obtain N first image features; extract image features from the M database images to obtain M second image features.

[0008] The N first image features are concatenated with the M second image features to obtain fused features. The fused features are then enhanced using a self-attention mechanism to obtain enhanced fused features.

[0009] From the enhanced fusion features, extract the N enhanced first image features corresponding to the N first image features and the M enhanced second image features corresponding to the M second image features. Based on the feature similarity between the N enhanced first image features and the M enhanced second image features, obtain an image that matches each human image to be identified.

[0010] In a second aspect, embodiments of the present invention provide a human image re-identification device, the human image re-identification device comprising:

[0011] The acquisition module is used to acquire N images of the human body to be identified, as well as M database images stored in the database, where N and M are both integers greater than zero;

[0012] The feature extraction module is used to extract image features from the N human images to be identified to obtain N first image features, and to extract image features from the M database images to obtain M second image features;

[0013] The fusion module is used to concatenate the N first image features with the M second image features to obtain fused features, and to enhance the fused features through a self-attention mechanism to obtain enhanced fused features;

[0014] The re-identification module is used to extract N enhanced first image features corresponding to the N first image features and M enhanced second image features corresponding to the M second image features from the enhanced fusion features, and obtain an image matching each human image to be identified based on the feature similarity between the N enhanced first image features and the M enhanced second image features.

[0015] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human image re-identification method as described in the first aspect.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the human image re-identification method as described in the first aspect.

[0017] The advantages of this invention compared to the prior art are:

[0018] N images of human bodies to be identified and M database images stored in a database are obtained, where N and M are both positive integers. Image features are extracted from the N images of human bodies to be identified to obtain N first image features. Image features are extracted from the M database images to obtain M second image features. The N first image features and the M second image features are concatenated to obtain fused features. Feature enhancement is performed on the fused features through a self-attention mechanism to obtain enhanced fused features. From the enhanced fused features, N enhanced first image features corresponding to the N first image features and M enhanced second image features corresponding to the M second image features are extracted. Based on the feature similarity between the N enhanced first image features and the M enhanced second image features, an image matching each image of human bodies to be identified is obtained. In this invention, the features of human bodies in the database are used to enhance the features of the image to be identified, and the enhanced features are used for recognition in the database. Because the features of human bodies in the database are used to enhance the features of the image to be identified, the correlation between the enhanced features and the features of human bodies in the database is improved, thereby improving the corresponding re-identification rate when re-identifying human bodies. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for a human image re-identification method provided in an embodiment of the present invention;

[0021] Figure 2 This is a flowchart illustrating a human image re-identification method according to an embodiment of the present invention;

[0022] Figure 3 This is a flowchart illustrating a human image re-identification method according to another embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of a human image re-recognition device provided in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0027] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0028] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0030] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0033] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0034] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0035] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0036] This invention provides a method for re-identifying human images, which can be applied to, for example... Figure 1In this application environment, the client communicates with the server. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0037] See Figure 2 This is a flowchart illustrating a human image re-identification method according to an embodiment of the present invention. The above-described human image re-identification method can be applied to... Figure 1 The server in the system is used to identify the corresponding human body image, such as Figure 2 As shown, the human image re-identification method may include the following steps:

[0038] S201: Obtain N images of the human body to be identified, and M database images stored in the database.

[0039] In step S201, the N human images to be identified can be obtained in various ways. For example, a visual sensor can be used to acquire images of the target scene as images to be identified, and M database images stored in the database can be used to retrieve target images that match the N human images to be identified. N and M are both integers greater than zero.

[0040] In this embodiment, the human image to be identified can be any suitable image that requires human re-identification, such as a video frame captured for a target area. The human image to be identified can be an original image captured by an image acquisition device such as a camera, or an image obtained after preprocessing the original image. This preprocessing operation can include noise reduction operations such as filtering. The N human images to be identified can come from the same image acquisition device or from different image acquisition devices. The M database images store human images of users, containing multiple user human images and corresponding identity information. The human image re-identification process determines the corresponding identity information of the human image to be identified based on the multiple user human images stored in the database. For example, pedestrian A and pedestrian B are two different human categories. In one example, for ease of recording, identity document (id) can be used to record and distinguish each human category. For example, id1 corresponds to the first human category, id2 corresponds to the first human category.

[0041] It should be noted that the image of the person to be identified can be a complete image or an incomplete image. For example, the image can show the upper body or the lower body of a person. There can be one or multiple images of the person to be identified. For instance, the image can be a single frame from a surveillance video, or it can be multiple frames from a surveillance video.

[0042] Databases used for storing images can utilize NAS (Network Attached Storage), a type of dedicated data storage server. NAS is data-centric, completely separating storage devices from servers and centrally managing data, thereby freeing up bandwidth, improving performance, reducing total cost of ownership, and protecting investment. Its cost is far lower than using server storage, while its efficiency is significantly higher, improving the efficiency and response speed of data exchange between services.

[0043] In this embodiment, the target image corresponding to the image to be identified is used by the image recognition stored in the database. The human image to be identified is input to the corresponding electronic device. The electronic device can determine whether the human image to be identified is the target image according to the human image re-identification method of this application embodiment.

[0044] S202: Extract image features from N images of the human body to be identified to obtain N first image features, and extract image features from M database images to obtain M second image features.

[0045] In step S202, image features are extracted from N images of the human body to be identified to obtain N first image features. Image features are extracted from M database images, wherein the number of features extracted from each image is equal, so that the first image features and the second image features can be spliced ​​together.

[0046] In this embodiment, the features in the N images of the human body to be identified and the M images in the database refer to unstructured human body features, i.e., feature vectors. For example, in a convolutional neural network (CNN) model, the feature output may be 256-dimensional, then the human body feature is a 256-dimensional vector, regardless of data type. However, each dimension may not represent any semantic information.

[0047] It should be noted that unstructured features can be understood as the overall appearance features. In other words, we do not need to care about the specific meaning of these features. For example, human body features are affected by clothing, height, etc., resulting in differences in human body features between individuals. However, human body features do not care which features correspond to clothing, which features correspond to height, etc.

[0048] For example, extracting image features from N images of the human body to be identified yields N first image features. in, The image features corresponding to the first image of the human body to be identified. The image features corresponding to the second image of the human body to be identified. Let N be the image features corresponding to the Nth human image to be identified. Extract image features from M database images to obtain M second image features. The image features corresponding to the first image in the database. The image features corresponding to the second image in the database. The image features corresponding to the Mth image in the database.

[0049] In this embodiment, image features of the human body image to be identified and images in the database are extracted so that the image features in the database can be used to enhance the features of the human body image to be identified, thereby improving the correlation between the human body image to be identified and the images in the database.

[0050] Optionally, image features are extracted from N images of the human body to be identified to obtain N first image features, and image features are extracted from M database images to obtain M second image features, including:

[0051] Using a convolutional neural network, N first image features and M second image features are extracted from N images of the human body to be identified and M database images, respectively.

[0052] In this embodiment, a convolutional neural network (CNN) is used to extract image features. A CNN is a deep neural network with a convolutional structure. A CNN includes a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer refers to the layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way image information is extracted being independent of location. The convolutional kernel can be initialized as a matrix of random size, and during the training of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0053] In convolutional neural networks (CNNs), the convolutional layer is the core module. Its primary function is to extract features from raw data. A convolutional layer takes features from the previous layer and a convolutional kernel, performs a convolution operation, and then applies an activation function to obtain the convolution result for that layer, thus forming its features. Shallow convolutional layers can only extract relatively low-level features, while deeper layers extract more complex pedestrian features from lower-level features. The features output by a convolutional layer are related to the convolution of several features from the previous layer. Each feature can be convolved using different convolutional kernels.

[0054] It should be noted that when using convolutional neural networks (CNNs) to extract image features, the CNN needs to be trained before using the trained CNN to extract image features. Training can be performed by extracting a human image re-identification dataset from a large public dataset, using median filtering to smooth the images in the dataset, resulting in a preprocessed dataset. This preprocessed dataset is then processed using a grayscale stretching image enhancement algorithm to obtain an enhanced dataset. An initial CNN is then trained based on this enhanced dataset, resulting in a trained CNN. N human images to be identified and M database images are then input into the trained CNN model, which outputs N first image features and M second image features.

[0055] In another embodiment, when using a convolutional neural network to extract image features, the obtained image features are local features. To facilitate the extraction of local features, image feature sequences corresponding to the human image to be identified and the database image are extracted based on the result of serialization processing of the human image to be identified and the database image. This image feature sequence is the initial image feature of the human image to be identified and the database image. Therefore, to enhance the expression of local features of the human image to be identified and the database image and improve the accuracy of human image to be identified and database image recognition, the image feature sequence is matrix-processed, converting the one-dimensional sequence into a matrix with a specified dimension, thereby obtaining the image feature vector corresponding to the human image to be identified and the database image. In one embodiment, the specified dimension can be two-dimensional. For example, if the image feature sequence is a 64*268 sequence, after performing 8*8 (8 rows and 8 columns) matrix processing on the human image to be identified and the database image, a two-dimensional image feature vector of 8*8*268 (an 8-row, 8-column matrix, where the data in each row and each column is 268-dimensional) is obtained. Furthermore, by using multiple convolutional layers with the same specified kernel size but different dimensions, the image feature vectors are convolved to extract local feature vectors of the human image to be identified and the database image in different dimensions. The specified kernel size of each convolutional layer is the same, which helps to ensure that the number of local features extracted from the human image to be identified and the database image is equal when extracting local feature vectors, thereby improving the fusion accuracy.

[0056] S203: Concatenate N first image features with M second image features to obtain fused features. Then, enhance the fused features through a self-attention mechanism to obtain enhanced fused features.

[0057] In step S203, the N first image features and M second image features are concatenated to obtain fused features. Through a self-attention mechanism, the correlation value between each feature in the fused features is calculated. The correlation value is used as a weight to enhance the fused features, resulting in enhanced fused features.

[0058] In this embodiment, N first image features obtained from feature extraction are concatenated with M second image features to obtain fused features. For example, the formula for concatenating N first image features and M second image features is as follows:

[0059]

[0060] Where f is the fusion feature, f1 is N first image features, and f2 is M second image features. The size of f1 is n×d, the size of f2 is m×d, and d is the dimension of the first image features and the second image. Therefore, the size of f is (n+m)×d.

[0061] In this embodiment, N first image features and M second image features are fused by splicing, so that the fused features can include all image features of the human image to be identified and the images in the database. This is so that when calculating the self-attention value, the attention value of each image feature can traverse all image features, taking into account the importance of all image features to the image feature, thereby obtaining a more accurate attention value.

[0062] After obtaining the fused features, a self-attention mechanism is used to enhance them. This feature enhancement is vector-based. The fused feature matrix, obtained by concatenating N first image features and M second image features, is then vectorized into a one-dimensional vector to meet the processing requirements of the subsequent self-attention mechanism. The resulting one-dimensional vector represents the local feature vector corresponding to the N first image features and M second image features in the current dimension.

[0063] This embodiment enhances the fused features through a self-attention mechanism, which fully explores the attention relationship between the human image to be identified and the images in the database, thereby obtaining more accurate enhanced features. This helps improve the robustness of the human image to be identified. The self-attention mechanism in this embodiment allows more resources to be allocated to the most information-rich and discriminative regions during feature extraction, achieving the goal of highlighting key information and suppressing irrelevant information, thus further extracting relevant information.

[0064] Optionally, the N first image features are concatenated with M second image features to obtain fused features. These fused features are then enhanced using a self-attention mechanism to obtain enhanced fused features, including:

[0065] The N first image features are concatenated with the M second image features to obtain the concatenated fused features. The number of fused features is the sum of the number of first image features and the number of second image features.

[0066] L image features are collected from the fused features to obtain the sub-features corresponding to the fused features;

[0067] Calculate the self-attention matrix of the fused feature and its sub-features, and perform feature enhancement on the fused feature based on the self-attention matrix to obtain the enhanced fused feature.

[0068] In this embodiment, the N first image features and M second image features are concatenated to obtain the concatenated fused features. During concatenation, both the N first image features and the M second image features are composed of feature matrices. The feature matrices corresponding to the M second image features are directly concatenated after the feature matrices corresponding to the N first image features to obtain the augmented feature matrix. The augmented feature matrix is ​​used as the fused feature matrix corresponding to the fused features.

[0069] The feature matrix corresponding to the fused feature f is obtained as follows When performing self-attention calculation on the fused features, the fused feature matrix The feature matrix in the original text is (n+m)×d, which results in a large computational load during self-attention calculation, reducing operational efficiency. Therefore, in this embodiment, L image features are collected from the fused features to obtain the corresponding sub-features. The self-attention matrix between the fused features and the sub-features is calculated, and feature enhancement is performed on the fused features based on the self-attention matrix to obtain the enhanced fused features. The L image features collected from the fused features are much smaller than (n+m). Using the collected L image features instead of the fused features for self-attention calculation significantly reduces the computational load during sub-attention calculation and improves the computational efficiency of self-attention calculation.

[0070] When collecting L image features from the fused features, any L image features can be collected without specifying the ratio of the number of first image features to the number of second image features. For example, the fused features can be numbered, with each image feature corresponding to a number. A random function can be used to randomly select L numbers, resulting in the L image features corresponding to the selected numbers. Alternatively, image features can be selected from N first image features and M second image features based on the ratio of the number of first image features to the number of second image features. For example, when the ratio of the number of first image features to the number of second image features is 1:5, the ratio of the number of first image features to the number of second image features collected from the fused features will also be 1:5. Other methods can also be used to collect the corresponding sub-features, which will not be described in detail.

[0071] In this embodiment, L image features are collected from the fused features to obtain the sub-features corresponding to the fused features. The self-attention matrix of the fused features and the sub-features is calculated. The fused features are enhanced according to the self-attention matrix to obtain the enhanced fused features. The self-attention of the fused features is transferred to the attention of the fused features and the corresponding sub-features. Since the sub-features are local image features collected from the fused features, using the sub-features to replace the fused features will greatly reduce the amount of computation when calculating the sub-attention values ​​and improve the computation rate of the self-attention calculation.

[0072] Optionally, the self-attention matrix of the fused features and sub-features is calculated, including:

[0073] Obtain the fusion feature matrix corresponding to the fusion feature, and perform spatial transformation on the fusion feature matrix using a preset transformation matrix corresponding to the dimension of the fusion feature matrix to obtain the first feature matrix set corresponding to the fusion feature;

[0074] Obtain the sub-feature matrix corresponding to the sub-feature, and perform spatial transformation on the sub-feature matrix using a preset transformation matrix corresponding to the dimension of the sub-feature matrix to obtain the second feature matrix set corresponding to the fused feature;

[0075] Using the feature matrices from the first and second feature matrix sets, calculate the self-attention matrix of the fused features and sub-features.

[0076] Specifically, when calculating the attention matrix, the input feature matrix needs to be spatially transformed to obtain a feature matrix of a preset dimension. In this embodiment, the input feature matrix includes a fused feature matrix and sub-feature matrices. Spatial transformations are performed on the fused feature matrix and the sub-feature matrices respectively to obtain spatially transformed feature matrices corresponding to the fused feature matrix and the sub-feature matrices. During the spatial transformation, different fully connected layers can be used to perform the spatial transformation on the fused feature matrix and the sub-feature matrix to obtain the corresponding transformed feature matrices. When using fully connected layers with different parameters to perform spatial transformation on the fused feature matrix or the sub-feature matrix, the resulting transformed feature matrices corresponding to the fused feature matrix or the sub-feature matrix will be different. Therefore, different fully connected layers can be used to perform multiple spatial transformations on the fused feature matrix or the sub-feature matrix to obtain a first feature matrix set corresponding to the fused feature matrix and a second feature matrix set corresponding to the sub-feature matrix. Self-attention is then calculated using the second feature matrix set corresponding to the fused feature matrix and the second feature matrix set corresponding to the sub-feature matrix.

[0077] In this embodiment, when performing spatial transformation on the fused feature matrix, three different fully connected layers are used to perform spatial transformation on the fused feature matrix. The transformation matrix in each fully connected layer is a preset transformation matrix corresponding to the dimension of the fused feature matrix, thus converting the fused matrix into three different transformation matrices, namely Q, K, and V. The size of the three transformation matrices Q, K, and V is (n+m)×d, and the first feature matrix set includes the three transformation matrices Q, K, and V. When performing spatial transformation on the sub-feature matrices, two different fully connected layers are used to perform spatial transformation on the sub-feature matrices. The transformation matrix in each fully connected layer is a preset transformation matrix corresponding to the dimension of the sub-feature matrix, thus converting the fused matrix into two different transformation matrices, namely k... l ,q l Two transformation matrices, kl ,q l The two transformation matrices are of size l×d, and the second characteristic matrix set includes k l ,q l Two transformation matrices.

[0078] After obtaining the corresponding transformation matrix sets from the fused feature matrix and the sub-feature matrix, self-attention values ​​are calculated. Typically, when calculating the self-attention value corresponding to the fused feature, two transformation matrices Q and K from the fused feature matrix are multiplied together to obtain an (n+m)×(n+m) matrix. The result of this multiplication is used as the weight value of the fused feature. This weight value is then multiplied with the transformation matrix V to obtain the matrix corresponding to the enhanced fused feature. However, the multiplication of the feature matrices Q and K involves a large amount of data, increasing the computational burden.

[0079] In this embodiment, the transformation matrix k corresponding to the sub-feature matrix is ​​used. l ,q l Reduce the size of the matrices, since the size of the three transformation matrices Q, K, and V is (n+m)×d, k l ,q l Since the two transformation matrices are of size l×d, they have the same dimension. We use the transformation matrix Q corresponding to the fused feature matrix and the transformation matrix k corresponding to the sub-feature matrix. l Perform matrix multiplication, and use the result of the multiplication as the new transformation matrix Q corresponding to the new fusion feature matrix. l The transformation matrix K corresponding to the fused feature matrix and the transformation matrix q corresponding to the sub-feature matrix are then combined. l Perform matrix multiplication, and use the result of the multiplication as the new transformation matrix K corresponding to the new fusion feature matrix. l Because of k l ,q l The two transformation matrices have a dimension of l×d, which is much smaller than the matrix dimension of Q,K, thus reducing the corresponding computational cost. Where Q... l With K l The size is (n+m)×l, and finally Q l With K l Perform matrix multiplication to obtain an (n+m)×(n+m) matrix. The final matrix retains the same size, using k as an intermediate step. l ,q l This involves reducing the matrix dimension, thereby reducing computational cost. The formula for calculating the attention matrix is ​​as follows:

[0080] A l =(Q⊙(k) l ) T )⊙(K⊙(q l )T ) T

[0081] Among them, A l Let Q be the self-attention matrix, and K be the transformation matrix corresponding to the fused feature matrix. l ,q l Let be the transformation matrix corresponding to the sub-feature matrix, (.). T ⊙ denotes matrix transpose, and ⊙ denotes matrix multiplication. In this embodiment, k is used when calculating the attention matrix. l ,q l Using intermediate operations can reduce the dimensionality of the operation, thereby reducing the amount of computation.

[0082] After obtaining the corresponding attention matrix, the self-attention matrix is ​​used to enhance the fused features. The self-attention matrix is ​​multiplied by another transformation matrix V obtained by spatial transformation of the fused feature matrix to obtain the feature matrix corresponding to the enhanced fused features.

[0083] In this embodiment, the enhanced fusion features are obtained through the self-attention matrix, the attention relationship between each feature in the fusion features and all image features is determined, and then each feature in the fusion features is weighted and accumulated to improve the expression of image features.

[0084] Optionally, the self-attention matrix of the fused feature and the sub-features is calculated, and the fused feature is enhanced based on the self-attention matrix to obtain the enhanced fused feature, including:

[0085] The self-attention matrix is ​​sparsified to obtain the corresponding sparsified matrix.

[0086] The fused features are enhanced by using a sparse matrix to obtain the enhanced fused features.

[0087] In this embodiment, the corresponding self-attention matrix A is obtained. l Afterwards, A of the self-attention matrix l The size is (n+m)×(n+m), the matrix dimension is large, so a self-attention matrix A is used. l To improve computational efficiency when performing weight multiplication, the self-attention matrix A is adjusted. l Sparsification is performed to facilitate the weight accumulation operation. After obtaining the sparsified matrix corresponding to the self-attention matrix, the fused features are enhanced using the sparsified matrix to obtain the enhanced fused features. The sparsified matrix is ​​then multiplied by the corresponding transformation matrix V to obtain the feature matrix corresponding to the enhanced fused features.

[0088] Optionally, the self-attention matrix is ​​sparsified to obtain a sparsified matrix corresponding to the self-attention matrix, including:

[0089] The values ​​in each row of the attention matrix are obtained, and the values ​​in each row are sorted in descending order to obtain the sorting result of each row.

[0090] Select k values ​​from the sorted results in each row in descending order, set the k values ​​in each row to 1, and set the remaining values ​​in each row to 0 to obtain the corresponding sparse matrix.

[0091] In this embodiment, when sparsifying the self-attention matrix, the values ​​in the self-attention matrix are sparsified to 1 and 0. When multiplying the sparsified matrix with the corresponding transformation matrix V, if the value is 1, the final result is the addition of values, avoiding multiplication operations and thus improving the computation rate of obtaining the enhanced fusion features. If the value is 0, computation is avoided, which also improves the computation rate of obtaining the enhanced fusion features. Therefore, when sparsifying the self-attention matrix, the corresponding values ​​in the self-attention matrix are sparsified to the corresponding 1 and 0.

[0092] When the self-attention matrix is ​​set to 1 or 0, the values ​​in the self-attention matrix are determined by the size of the values. The first k largest values ​​in each row of the self-attention matrix are selected and set to 1, while the remaining values ​​are set to 0. This process involves obtaining the values ​​in each row of the self-attention matrix, sorting the values ​​in each row in descending order, and then selecting k values ​​from the sorted results in each row in descending order. These k values ​​are then set to 1, and the remaining values ​​in each row are set to 0, resulting in the corresponding sparsification matrix.

[0093] S204: Extract N enhanced first image features corresponding to N first image features and M enhanced second image features corresponding to M second image features from the enhanced fusion features. Based on the feature similarity between the N enhanced first image features and the M enhanced second image features, obtain an image that matches each human image to be identified.

[0094] In step S204, the target image matching the human image to be identified is obtained by searching the database according to the corresponding enhanced fusion features. The enhanced fusion features include N enhanced first image features corresponding to the N first image features and M enhanced second image features corresponding to the M second image features. The enhanced first image features and enhanced second image features are matched. After successful matching, the re-identification result is obtained.

[0095] In this embodiment, N enhanced first image features corresponding to N first image features and M enhanced second image features corresponding to M second image features are extracted from the enhanced fusion features. For example, the enhanced fusion features are... The first n features are extracted from the enhanced fusion features F to form N enhanced first image features, and the last m features are M enhanced second image features.

[0096] Using N enhanced first image features, find the corresponding matching enhanced second image features from M enhanced second image features. Use the image corresponding to the matching enhanced second image features as the image matched with the human body image to be identified. For example, using the first enhanced first image features... When searching for the corresponding matched enhanced second image features, Each of the M enhanced second image features Perform a one-to-one matching process, calculate the feature similarity between the first image feature and M enhanced second image features, obtain the enhanced second image feature corresponding to a successful match based on the similarity, and then find the corresponding image based on the number of the corresponding enhanced second image feature.

[0097] In this embodiment, enhanced features are used for re-identification. The enhanced first image feature includes the correlation between the image features in the database and the human image to be identified, and the enhanced second image feature includes the correlation between the human image features to be identified and the images in the database. Therefore, when using the enhanced first image feature and the enhanced second image feature for re-identification, the corresponding re-identification rate can be improved.

[0098] Optionally, based on the feature similarity between N enhanced first image features and M enhanced second image features, an image matching each human body image to be identified is obtained, including:

[0099] Calculate the similarity between each enhanced first image feature and M enhanced second image features;

[0100] The image with the highest similarity is identified as the image that matches the corresponding human body image to be identified.

[0101] In this embodiment, when re-identifying each human image to be identified, the similarity between each enhanced first image feature and M enhanced second image features is calculated, and the image corresponding to the one with the highest similarity is determined as the image that matches the human image to be identified, thus obtaining the re-identification result.

[0102] It should be noted that when determining the maximum similarity, the selection can be directly based on the similarity value between each enhanced first image feature and M enhanced second image features. If the database does not store the corresponding image to be identified, there will be a maximum similarity value when calculating the similarity between the enhanced first image feature and the M enhanced second image features. If the database image corresponding to the maximum similarity value is determined as the image matching the corresponding human image to be identified, the re-identification result will fail. Therefore, during re-identification, a corresponding threshold can be set. When the maximum similarity value is greater than the preset threshold, the image corresponding to the maximum similarity value is determined as the image matching the human image to be identified, and the re-identification result is obtained. If the maximum similarity value is not greater than the preset threshold, it is considered that there is no corresponding image to be identified in the database. Therefore, when determining the database image corresponding to the maximum similarity value, the corresponding database image can be used as a candidate target image. When the maximum similarity value is greater than the preset threshold, the candidate target image is used as the final image matching the corresponding human image to be identified.

[0103] For example, when the first enhanced first image features With M enhanced second image features Perform a one-to-one matching, calculate the feature similarity between the first image feature and the M enhanced second image features, and if the enhanced second image feature corresponding to the maximum similarity is... Then the enhanced second image features The corresponding database image is used as the candidate target image g m The maximum similarity value is compared with a preset threshold. When the feature similarity is greater than the preset threshold, the candidate target image g is selected. m As an image that is matched with the corresponding human body image to be identified, if the feature similarity is not greater than a preset threshold, the database does not store the corresponding human body image to be identified.

[0104] In this embodiment, when determining the image that matches the corresponding human body image to be identified, a corresponding threshold is set. The maximum similarity value and the threshold are used as conditions for determining the image that matches the corresponding human body image to be identified. This avoids the recognition error caused by using the maximum similarity value as a single condition to determine the image that matches the corresponding human body image to be identified.

[0105] It is understood that, in the specific embodiments of this application, the data related to human images, etc., requires user permission or consent when the embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0106] Please see Figure 3 , Figure 3This is a flowchart illustrating a human image re-identification method according to another embodiment of the present invention, as shown below. Figure 3 In this human image re-identification method, the step of identifying a target image matching the human image to be identified based on the feature similarity between the N enhanced first image features and the M enhanced second image features, and obtaining the re-identification result, includes:

[0107] S301: Calculate the similarity between each enhanced first image feature and the M enhanced second image features;

[0108] S302: Determine the image corresponding to the enhanced second image feature whose similarity is greater than the first preset threshold as the candidate target image;

[0109] S303: Calculate the average pixel value of the image to be identified corresponding to the enhanced first image feature and the average pixel value of each image in the candidate target image. When the difference between the average pixel value of the image to be identified and the average pixel value of the candidate target image is the smallest, the corresponding candidate target image is determined as the image that matches the corresponding human body image to be identified.

[0110] As an example, after calculating the similarity value between each enhanced first image feature and M enhanced second image features, a corresponding first preset threshold can be set. The similarity value is compared with the first preset threshold. When there is a similarity greater than the first preset threshold, multiple candidate target images are selected according to the set second preset threshold. The database image corresponding to the enhanced second image feature with a similarity greater than the second preset threshold is taken as the candidate target image. When selecting an image from the candidate target images that matches the corresponding human image to be identified, the pixel mean of the image to be identified and the candidate target images is calculated. When the difference between the pixel mean of the image to be identified and the candidate target images is the smallest, the corresponding candidate target image is taken as the image that matches the corresponding human image to be identified.

[0111] In this embodiment, the average pixel value of the image to be identified and the candidate target image is used as the condition for selecting the image that matches the corresponding human body image to be identified. The pixels of the image to be identified and the candidate target image are all pixel values ​​in the original image, which avoids the cumulative error caused by the feature extraction process and thus improves the re-identification rate.

[0112] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a human image re-recognition device provided in an embodiment of the present invention. In this embodiment, the terminal includes units used for performing... Figure 2 The steps in the corresponding embodiments. Please refer to the details. Figure 2 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 4The human image re-recognition device 40 includes: an acquisition module 41, a feature extraction module 42, a fusion module 43, and a re-recognition module 44.

[0113] The acquisition module 41 is used to acquire N images of the human body to be identified, and M database images stored in the database, where N and M are both integers greater than zero.

[0114] The feature extraction module 42 is used to extract image features from N images of the human body to be identified, to obtain N first image features, and to extract image features from M database images, to obtain M second image features.

[0115] The fusion module 43 is used to concatenate N first image features with M second image features to obtain fused features. Through a self-attention mechanism, the fused features are enhanced to obtain enhanced fused features.

[0116] The re-identification module 44 is used to extract N enhanced first image features corresponding to N first image features and M enhanced second image features corresponding to M second image features from the enhanced fusion features, and obtain an image matching each human image to be identified based on the feature similarity between the N enhanced first image features and the M enhanced second image features.

[0117] Optionally, the feature extraction module 42 described above includes:

[0118] The convolutional unit is used to extract N first image features and M second image features corresponding to N human images to be identified and M database images, respectively, using a convolutional neural network.

[0119] Optionally, the aforementioned fusion module 43 includes:

[0120] The stitching unit is used to stitch together the N first image features and M second image features to obtain the stitched fused features. The number of fused features is the sum of the number of first image features and the number of second image features.

[0121] The acquisition unit is used to acquire L image features from the fused features to obtain the sub-features corresponding to the fused features.

[0122] The feature enhancement unit is used to calculate the self-attention matrix of the fused feature and the sub-features, and to enhance the fused feature based on the self-attention matrix to obtain the enhanced fused feature.

[0123] Optionally, the aforementioned feature enhancement unit includes:

[0124] The first spatial transformation subunit is used to obtain the fusion feature matrix corresponding to the fusion feature, and to perform spatial transformation on the fusion feature matrix using a preset transformation matrix corresponding to the dimension of the fusion feature matrix to obtain the first feature matrix set corresponding to the fusion feature.

[0125] The second spatial transformation subunit is used to obtain the sub-feature matrix corresponding to the sub-feature, and to perform spatial transformation on the sub-feature matrix using a preset transformation matrix corresponding to the dimension of the sub-feature matrix to obtain the second feature matrix set corresponding to the fused feature.

[0126] The computational subunit is used to compute the self-attention matrix of the fused features and sub-features using the feature matrices in the first feature matrix set and the second feature matrix set.

[0127] Optionally, the aforementioned feature enhancement unit includes:

[0128] The sparsification subunit is used to sparsify the self-attention matrix to obtain the sparsified matrix corresponding to the self-attention matrix.

[0129] The enhanced fusion feature determines the sub-unit, which is used to enhance the fusion feature using a sparse matrix to obtain the enhanced fusion feature.

[0130] Optionally, the above-mentioned sparsification subunit includes:

[0131] The sorting unit is used to obtain the value of each row in the self-attention matrix, sort the values ​​in each row in descending order, and obtain the sorting result of each row.

[0132] The sparsification matrix determines the sub-units, which are used to select k values ​​from the sorted results in each row in descending order, set the k values ​​in each row to 1, and set the remaining values ​​in each row to 0, thus obtaining the corresponding sparsification matrix.

[0133] Optionally, the aforementioned re-identification module 44 includes:

[0134] The similarity calculation unit is used to calculate the similarity between each enhanced first image feature and M enhanced second image features.

[0135] The target image determination unit is used to determine the image with the highest similarity as the image that matches the corresponding human body image to be identified.

[0136] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0137] Figure 5This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executing the computer program, implements the steps in any of the above embodiments of the human image re-identification method.

[0138] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0139] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0140] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0141] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0142] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0145] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for re-identifying human images, characterized in that, The human image re-identification method includes: Obtain N human images to be identified, and M database images stored in the database, where N and M are both integers greater than zero; Extract image features from the N images of the human body to be identified to obtain N first image features; extract image features from the M database images to obtain M second image features. The N first image features are concatenated with the M second image features to obtain fused features. The fused features are then enhanced using a self-attention mechanism to obtain enhanced fused features. From the enhanced fusion features, extract the N enhanced first image features corresponding to the N first image features and the M enhanced second image features corresponding to the M second image features. Based on the feature similarity between the N enhanced first image features and the M enhanced second image features, obtain an image that matches each human image to be identified.

2. The human image re-identification method as described in claim 1, characterized in that, The process involves extracting image features from the N images of the human body to be identified, yielding N first image features, and extracting image features from the M database images, yielding M second image features, including: Using a convolutional neural network, N first image features and M second image features are extracted from N human images to be identified and M database images, respectively.

3. The human image re-identification method as described in claim 1, characterized in that, The step of concatenating the N first image features with the M second image features to obtain fused features, and then enhancing the fused features through a self-attention mechanism to obtain enhanced fused features, includes: The N first image features are concatenated with the M second image features to obtain the concatenated fused features. The number of fused features is the sum of the number of first image features and the number of second image features. L image features are collected from the fusion features to obtain the sub-features corresponding to the fusion features; Calculate the self-attention matrix of the fused feature and the sub-feature, and perform feature enhancement on the fused feature based on the self-attention matrix to obtain the enhanced fused feature.

4. The human image re-identification method as described in claim 3, characterized in that, The calculation of the self-attention matrix of the fused feature and the sub-feature includes: Obtain the fusion feature matrix corresponding to the fusion feature, and perform spatial transformation on the fusion feature matrix using a preset transformation matrix corresponding to the dimension of the fusion feature matrix to obtain the first feature matrix set corresponding to the fusion feature; Obtain the sub-feature matrix corresponding to the sub-feature, and perform spatial transformation on the sub-feature matrix using a preset transformation matrix corresponding to the dimension of the sub-feature matrix to obtain the second feature matrix set corresponding to the fused feature; Using the feature matrices in the first feature matrix set and the second feature matrix set, calculate the self-attention matrix of the fused feature and the sub-feature.

5. The human image re-identification method as described in claim 3, characterized in that, The process of calculating the self-attention matrix of the fused feature and the sub-features, and performing feature enhancement on the fused feature based on the self-attention matrix to obtain the enhanced fused feature includes: The self-attention matrix is ​​sparsified to obtain the sparsified matrix corresponding to the self-attention matrix; The sparse matrix is ​​used to enhance the fused features, resulting in enhanced fused features.

6. The human image re-identification method as described in claim 5, characterized in that, The step of sparsifying the self-attention matrix to obtain the corresponding sparsified matrix includes: Obtain the value of each row in the self-attention matrix, sort the values ​​in each row in descending order, and obtain the sorting result of each row. Select k values ​​from the sorted results in each row in descending order, set the k values ​​in each row to 1, and set the remaining values ​​in each row to 0 to obtain the corresponding sparse matrix.

7. The human image re-identification method as described in claim 1, characterized in that, The step of obtaining an image matching each human body image to be identified based on the feature similarity between the N enhanced first image features and the M enhanced second image features includes: Calculate the similarity between each enhanced first image feature and the M enhanced second image features; The image with the highest similarity is determined as the image that matches the corresponding human body image to be identified.

8. A human image re-recognition device, characterized in that, The human image re-identification device includes: The acquisition module is used to acquire N images of the human body to be identified, as well as M database images stored in the database, where N and M are both integers greater than zero; The feature extraction module is used to extract image features from the N human images to be identified to obtain N first image features, and to extract image features from the M database images to obtain M second image features; The fusion module is used to concatenate the N first image features with the M second image features to obtain fused features, and to enhance the fused features through a self-attention mechanism to obtain enhanced fused features; The re-identification module is used to extract N enhanced first image features corresponding to the N first image features and M enhanced second image features corresponding to the M second image features from the enhanced fusion features, and obtain an image matching each human image to be identified based on the feature similarity between the N enhanced first image features and the M enhanced second image features.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human image re-identification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the human image re-identification method as described in any one of claims 1 to 7.

Images