A person re-identification method, device, equipment and medium

Abstract

The application discloses a person re-identification method, device and equipment and medium, and belongs to the technical field of computer vision monitoring. The method comprises the following steps: obtaining a visual feature vector through a visual encoder based on a to-be-identified image of a target object; performing cross-modal similarity calculation based on the visual feature vector and a plurality of predefined semantic labels, obtaining a semantic attribute probability distribution, and mapping the semantic attribute probability distribution into a semantic embedding vector; obtaining a final identity embedding vector through foreground saliency decoupling based on the visual encoder, the visual feature vector and the semantic embedding vector; and performing similarity retrieval in a preset identity feature library based on the final identity embedding vector, and determining an identification result according to a retrieval result. Through the application, the technical problem of low precision of cross-camera person re-identification in an industrial complex scene with high homogeneity in appearance is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision monitoring, and particularly relates to a person re-identification method and device, equipment and a medium. BACKGROUND

[0002] Person re-identification is one of the core technologies of an intelligent video monitoring system, and aims to determine whether the personnel images detected by different cameras at different times belong to the same identity in a non-overlapping multi-camera network.

[0003] In the prior art, a person re-identification method based on pure visual features uses a convolutional neural network (CNN) or a vision transformer (ViT) as a backbone network to extract a global visual feature vector of personnel, and performs identity matching through cosine similarity or Euclidean distance, without distinguishing foreground and background regions and without introducing semantic attribute information.

[0004] This method has good performance when processing personnel groups with obvious appearance differences. However, the pure visual feature method has low accuracy in appearance highly homogenized scenes, and cannot realize high-precision cross-camera person re-identification in appearance highly homogenized industrial complex scenes. SUMMARY

[0005] The application provides a person re-identification method, device, equipment and medium, which can solve the technical problem of low precision of cross-camera person re-identification in appearance highly homogenized industrial complex scenes in the prior art.

[0006] To achieve the above object, the application provides the following technical solutions: In a first aspect, the application provides a person re-identification method, which comprises: Based on a to-be-identified image of a target object, a visual feature vector of the to-be-identified image is obtained through a visual encoder; Based on the visual feature vector and a plurality of predefined semantic labels, cross-modal similarity calculation is performed to obtain a semantic attribute probability distribution, and the semantic attribute probability distribution is mapped into a semantic embedding vector; Based on the visual encoder, the visual feature vector and the semantic embedding vector, a final identity embedding vector is obtained through foreground saliency decoupling; Based on the final identity embedding vector, similarity retrieval is performed in a preset identity feature library, and an identification result is determined according to the retrieval result.

[0007] Optionally, the visual encoder employs a pre-trained feature extraction backbone network, in which lightweight adapter modules are embedded in multiple network layers. The adapter module includes a dimension reduction projection layer, a non-linear activation layer, and a dimension increase projection layer, and the output of the adapter module is added to the original output of the network layer through residual connections. During training, the original parameters of the backbone network are frozen, and only the internal parameters of the adapter module and the learnable scaling factor of the adapter module are updated.

[0008] Optionally, the step of performing cross-modal similarity calculation based on the visual feature vector and multiple predefined semantic labels to obtain the semantic attribute probability distribution includes: obtaining the text description corresponding to each semantic label; encoding the text description corresponding to each semantic label into a corresponding text feature vector through a frozen text encoder to form a text feature vector set; mapping the visual feature vector to the space where the text feature vector set is located to obtain the mapped visual feature; calculating the similarity between the mapped visual feature and each text feature vector in the text feature vector set; generating the probability distribution of the image to be identified on each semantic label based on the similarity, and using it as the semantic attribute probability distribution.

[0009] Optionally, the step of obtaining the final identity embedding vector based on the visual encoder, the visual feature vector, and the semantic embedding vector through foreground saliency decoupling includes: generating a saliency map based on the attention weights within the visual encoder; correcting the saliency map using foreground segmentation constraints to obtain a foreground saliency map; weighting and aggregating the visual feature vector using the foreground saliency map to obtain global features for foreground enhancement; dividing the foreground region of the image to be identified into multiple sub-regions according to the human body structure, weighting and aggregating the visual features within each sub-region, and extracting local features from each sub-region; fusing the global features with each of the local features to obtain multi-granularity visual features; and performing cross-modal fusion of the multi-granularity visual features with the semantic embedding vector to generate the final identity embedding vector for identity determination.

[0010] Optionally, generating a saliency map based on the attention weights within the visual encoder includes: extracting the attention weights of the classification tokens to each image block token in the last layer of the visual encoder; averaging the results of each attention head; and mapping the averaged attention weights back to the spatial dimensions of the image to be recognized to obtain a saliency map.

[0011] Optionally, the step of modifying the saliency map using foreground segmentation constraints to obtain a foreground saliency map includes: receiving the feature map output by the intermediate layer of the visual encoder through a lightweight auxiliary network to obtain a foreground probability map output by the auxiliary network; wherein the auxiliary network is pre-trained using a semantic segmentation mask automatically generated by a pre-trained human parsing model as a supervision signal; and multiplying the foreground probability map and the saliency map element-wise to obtain the foreground saliency map.

[0012] Optionally, the step of performing a similarity search in a preset identity feature library based on the final identity embedding vector and determining the recognition result based on the search results includes: performing a coarse search using an approximate nearest neighbor index structure to obtain a candidate identity set from the preset identity feature library; calculating the similarity between the corresponding identity embedding vector and the final identity embedding vector for each identity in the candidate identity set; filtering and sorting the candidate identity set in combination with spatiotemporal constraints; and outputting the identity with the highest matching degree as the recognition result; wherein the spatiotemporal constraints include: determining whether the time difference between two captures of the person to be identified by different cameras and the spatial distance between the corresponding cameras is within a reasonable time window determined by the range of the person's movement speed and the path detour coefficient; and verifying whether the movement path of the person to be identified between the cameras is physically reachable based on a pre-built camera topology map.

[0013] Optionally, after performing a similarity search in a preset identity feature library based on the final identity embedding vector and determining the identification result based on the search results, the process includes: querying the permission whitelist corresponding to the security zones divided by the electronic fence system based on the identification result, determining whether the identity corresponding to the identification result has the permission to enter the current area, and detecting abnormal behavior patterns based on the spatiotemporal trajectory of the identity corresponding to the identification result under multiple cameras; if not, generating a boundary crossing alarm and pushing the boundary crossing alarm or the abnormal behavior pattern to the security management platform; wherein, the abnormal behavior patterns include prolonged stay in non-work areas, repeated entry and exit of restricted areas, reverse movement in one-way channels, and exceeding the maximum allowed working time in designated areas.

[0014] Secondly, embodiments of this application provide a person re-identification device, the device comprising: The visual encoding module is configured to obtain the visual feature vector of the image to be identified based on the image of the target object through a visual encoder. The semantic alignment module is configured to perform cross-modal similarity calculation based on the visual feature vector and multiple predefined semantic labels, obtain the semantic attribute probability distribution, and map the semantic attribute probability distribution into a semantic embedding vector; The foreground decoupling module is configured to obtain the final identity embedding vector based on the visual encoder, the visual feature vector, and the semantic embedding vector through foreground saliency decoupling. The retrieval and matching module is configured to perform a similarity search in a preset identity feature library based on the final identity embedding vector, and determine the recognition result based on the search results.

[0015] Based on the same inventive concept, this application also provides an electronic device, including: a memory and a processor; the processor is used to read and execute a computer program stored in the memory to implement the steps of the aforementioned person re-identification method.

[0016] Based on the same inventive concept, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement the steps of the aforementioned person re-identification method.

[0017] Compared with the prior art, this application has the following advantages: Based on the image to be identified of the target object, a visual feature vector of the image to be identified is obtained through a visual encoder; based on the visual feature vector and multiple predefined semantic tags, cross-modal similarity calculation is performed to obtain the semantic attribute probability distribution, and the semantic attribute probability distribution is mapped to a semantic embedding vector; the introduction of semantic tags provides a discrimination signal independent of appearance, opening up a new discrimination dimension for homogeneous appearances; Based on the visual encoder, the visual feature vector, and the semantic embedding vector, the final identity embedding vector is obtained through foreground saliency decoupling. Through foreground saliency decoupling, a complementary description of "overall outline + key parts" is achieved, ensuring the robustness of identity representation. Based on the final identity embedding vector, a similarity search is performed in a preset identity feature database, and the recognition result is determined according to the search results. By performing a similarity search in the preset identity feature database, high-precision, real-time cross-camera identity matching and security management are achieved.

[0018] The above steps work together to form a low-cost, high-precision, robust, and deployable complete industrial personnel re-identification solution, which solves the technical problem of low accuracy in cross-camera personnel re-identification in complex industrial scenarios with highly homogeneous appearances, which exists in existing technologies.

[0019] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of the personnel re-identification method provided in the embodiments of this application; Figure 2 A schematic diagram illustrating the collaborative relationship architecture of each functional module in the personnel re-identification method provided in this application embodiment; Figure 3 This is a schematic diagram of the functional modules of one embodiment of the personnel re-identification device of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] To address the shortcomings of existing technologies, refer to Figure 1 This application discloses a method for re-identifying persons, the method comprising: Step S10: Based on the image of the target object to be identified, obtain the visual feature vector of the image to be identified through a visual encoder; In this embodiment, an initial image of the target object to be identified is obtained, and the initial image to be identified is preprocessed such as size adjustment and pixel normalization to obtain the image of the target object to be identified.

[0024] The 256×128 pixel image is divided into non-overlapping image patches of a fixed size of 16×16 pixels. There are 256 / 16=16 patches in the height direction and 128 / 16=8 patches in the width direction, for a total of 16×8=128 image patches.

[0025] Each image patch originally has dimensions of 16×16×3=768. After flattening each image patch into a one-dimensional vector, a learnable linear projection layer (i.e., a Patch Embedding layer) maps each 768-dimensional patch vector to a 768-dimensional embedding space. This linear projection layer is essentially a two-dimensional convolutional layer with a kernel size of 16×16, a stride of 16, and 768 output channels. Through this mapping, the 128 image patches correspond to 128 768-dimensional patch tokens.

[0026] A learnable classification token (CLS Token), also with a dimension of 768, is appended before the 128 image patch tokens. The CLS Token aggregates global information during the Transformer encoding process, and its final output is used as a global representation of the image. Simultaneously, to preserve the spatial location information of the image patches, a learnable position embedding is superimposed on each token in the sequence (containing the CLS Token and 128 Patch Tokens). The position embedding dimension is (129)×768, matching the dimension of the token sequence.

[0027] After the above processing, the token sequence length of the input visual encoder is 129 (1 CLS Token + 128 Patch Tokens), and the embedding dimension of each token is 768.

[0028] The token sequence is encoded sequentially through 12 Transformer blocks of ViT-B / 16. Each Transformer block contains a Multi-Head Self-Attention (MHSA) layer and a Feed-Forward Network (FFN) layer, both preceded by Layer Normalization and followed by addition to the input via residual connections.

[0029] The model parameter configurations for ViT-B / 16 are shown in Table 1.

[0030] Table 1

[0031] In the standard CLIP ViT-B / 16 forward propagation, the output of the CLS Token after 12 layers of Transformer encoding (768 dimensions) is directly used as the visual feature vector of the image to be recognized.

[0032] Furthermore, this embodiment embeds a lightweight adapter module in each Transformer block of the visual encoder.

[0033] The adapters are embedded as follows: one adapter module is inserted after the multi-head self-attention layer and one after the feedforward network layer of each Transformer block. For 12 Transformer blocks, a total of 24 adapter modules are embedded.

[0034] Adapter bottleneck structure: Each adapter module adopts a "bottleneck" structure design, containing three components: Dimensionality reduction projection layer The input 768-dimensional features are compressed to 96 dimensions, with a dimensionality reduction ratio r = 768 / 96 = 8. Nonlinear activation layer: The GELU activation function is used, GELU(x)=x×Φ(x), where Φ is the cumulative distribution function of the standard normal distribution; Upward projection layer : Restore 96-dimensional features to 768-dimensional features.

[0035] The forward computation of the adapter is:

[0036] in, and This is a bias term. and Initialize to a small random value of U(-0.01, 0.01), and initialize the bias to zero to ensure that the adapter is approximately identically mapped in the initial state.

[0037] Wherein, Adapter(x) represents the mapping result of the adapter module to the input feature x; x represents the feature vector input to the adapter module, which has a dimension of 768; This represents the weight matrix of the dimension reduction projection layer, which has a dimension of 96×768; This represents the weight matrix of the up-dimensional projection layer, with dimensions of 768×96; This represents the bias vector of the dimension reduction projection layer, which has a dimension of 96; represents the bias vector of the up-dimensional projection layer, with a dimension of 768; GELU(·) represents the Gaussian error linear unit activation function.

[0038] Residual connections and scaling factors: The adapter's output is added to the original output of the network layer through residual connections.

[0039] in, The learnable scaling factor is initialized to 0.1 to ensure that the adapter's perturbation of the original features is sufficiently small during the initial training phase. Each adapter module is equipped with an independent... The parameters consist of 24 learnable scalars.

[0040] in, This represents the original output feature of the l-th network layer in the visual encoder; This indicates the output characteristics of the network layer after correction by the adapter module; (·) indicates insertion at the first... The adapter module after the network layer; This indicates the learnable scaling factor corresponding to the adapter module; The value ranges from 1 to 24.

[0041] Parameter count and training strategy: The number of parameters for a single adapter is 2 × 768 × 96 + 96 + 768 = 148,320. The total number of parameters for 24 adapters is approximately 148,320 × 24 = 3,559,680 (approximately 3.56 million), which is only 4.1% of the original parameters of ViT-B / 16 (approximately 86 million). During training, all the original parameters of ViT-B / 16 (approximately 86 million) are completely frozen, and only the internal parameters of the 24 adapters (approximately 3.56 million) and the 24 scaling factors are updated. .

[0042] Visual Feature Vector Output: After the token sequence passes through 12 Transformer blocks embedded with adapter modules, the output of the CLS Token (dimension 768) serves as the global visual feature vector of the image to be recognized. .

[0043] The training data was constructed using a dataset specifically designed for industrial scenarios. Data collection covered six cameras in the mining area, four time periods (6-8 AM, 11 AM-1 PM, 5-7 PM, and 10-2 AM), and four weather conditions (sunny, cloudy, dusty, and rainy / foggy), totaling 16 environmental combinations. Samples were collected for each identity in at least four environmental combinations to ensure cross-conditional generalization of feature learning. A total of 500+ identities and 30,000+ images were collected. Data augmentation strategies include four categories: dust simulation (overlaying Gaussian noise specks with U(0.1,0.4) opacity, radius 3-15px, number 50-200, probability 0.5), illumination variation (brightness ±30%, contrast ±30%, gamma γ∈[0.7,1.5], probability 0.8), random occlusion (50-150px rectangular color blocks, area ratio 10%-40%, probability 0.3), and color dithering (hue ±10°, saturation ±20%, brightness ±15%, probability 0.6). The training and validation sets are split by identity in an 8:2 ratio to ensure that the same identity does not appear in multiple sets. During training, the original parameters of the backbone network are frozen, and only the internal parameters of the adapter modules and the learnable scaling coefficients of the adapter modules are updated.

[0044] The visual feature vector v is then input to the subsequent cross-modal semantic alignment module and foreground saliency decoupling module, which are used to: calculate cross-modal similarity with the text features of the semantic label to generate a semantic attribute probability distribution, and generate a saliency map based on the attention weights of each layer of ViT for background interference suppression.

[0045] Step S20: Based on the visual feature vector and multiple predefined semantic labels, perform cross-modal similarity calculation to obtain the semantic attribute probability distribution, and map the semantic attribute probability distribution into a semantic embedding vector; In some specific embodiments, the step of calculating cross-modal similarity based on the visual feature vector and multiple predefined semantic labels to obtain the semantic attribute probability distribution includes: Obtain the text description corresponding to each semantic tag; The text descriptions corresponding to each semantic tag are encoded into corresponding text feature vectors by a frozen text encoder, forming a set of text feature vectors; The visual feature vectors are mapped to the space containing the text feature vector set to obtain the mapped visual features; Calculate the similarity between the mapped visual features and each text feature vector in the text feature vector set; The probability distribution of the image to be identified on each semantic label is generated based on the similarity, and used as the probability distribution of semantic attributes.

[0046] In this embodiment, an attribute description system with 22 semantic tags in four basic dimensions and 10 additional extended tag positions, totaling 32 semantic tags, is pre-constructed around the distinguishable attributes of personnel in industrial scenarios (such as mining areas, chemical plants, ports, etc.).

[0047] (1) Safety Equipment Dimension (7 tags): This describes the head protection and reflective markings on the body of personnel, including wearing a white safety helmet, wearing a red safety helmet, wearing a yellow safety helmet, wearing a blue safety helmet, not wearing a safety helmet, wearing a reflective vest, and not wearing a reflective vest. Among them, the color of the safety helmet corresponds to different job levels in the mining area scenario—white corresponds to management, red to technical personnel, yellow to ordinary workers, and blue to visitors. This color difference can serve as a strong clue for identity identification.

[0048] (2) Clothing characteristics dimension (5 tags): used to describe the type of work clothing worn by personnel, including wearing blue work clothes, wearing gray work clothes, wearing orange work clothes, wearing casual clothes, and wearing management uniforms.

[0049] (3) Body shape characteristics dimension (5 labels): used to describe the body shape of the person, including tall (height > 180cm), medium (height 165-180cm), short (height < 165cm), overweight (BMI > 25), and underweight (BMI < 20).

[0050] (4) Carried Items Dimension (5 tags): Used to describe a person's personal belongings and behavioral status, including carrying a tool bag, carrying a folder, riding an electric bike, pushing a trolley, and empty-handed.

[0051] The above four dimensions contain a total of 22 semantic tags, forming a basic tag set. An additional 10 extended tag positions are reserved for on-site customization based on specific scenarios (such as "wearing goggles" or "carrying a walkie-talkie"). The basic tags and extended tag positions together constitute 32 semantic tags, which correspond to the subsequent 32-dimensional semantic attribute probability distribution and 32×512-dimensional text feature matrix.

[0052] To adapt to the input format of the CLIP text encoder, at least one natural language description template is constructed for each semantic tag. The template format is: "A person in industrial site, [specific attribute description]".

[0053] The text descriptions of the 32 semantic tags mentioned above are input into the pre-trained CLIP text encoder for encoding. This text encoder is a 12-layer Transformer architecture with an output dimension of 512.

[0054] Before training begins, a one-time text feature extraction and caching operation is performed. Specifically, the 32 text descriptions are forward-propagated through a text encoder to obtain 32 512-dimensional text feature vectors. (Where k=1,2,...,32). All text feature vectors are stacked along the row direction to form a fixed text feature matrix. : .

[0055] The text feature matrix T remains constant throughout all subsequent training and inference processes (because the text encoder is frozen and gradients are not backpropagated to it). In actual engineering deployments, this matrix is ​​stored in GPU memory in NumPy or PyTorch tensor format for direct access during each forward propagation, avoiding the time overhead of repeated computation.

[0056] The CLS Token feature vector output by the visual encoder is Since this feature resides in the 768-dimensional space of the visual modality, which does not match the 512-dimensional space of the text feature, similarity calculation cannot be performed directly. This embodiment utilizes a learnable linear projection layer. Map visual features to the same common embedding space as text features: .

[0057] in, The projection layer The adapter parameters are updated along with the training process, and are initialized using a truncated normal distribution with a standard deviation of 0.02. After training converges, With all 32 text feature vectors Being located in the same 512-dimensional joint embedding space makes the cosine similarity between the two comparable.

[0058] It is worth noting that a single linear projection matrix is ​​used here. Instead of setting a projection matrix for each text label, the visual features of all training samples are uniformly projected to ensure that the position of the projected visual features in the public space is consistent and interpretable.

[0059] Visual features after projection Then, calculate its relationship with each text feature vector in the text feature matrix T. The cosine similarity.

[0060] The range of cosine similarity is [ [1, 1], the larger the value, the closer the visual features of the image are to the textual description of the corresponding semantic label in the joint embedding space.

[0061] Subsequently, the 32 similarity values ​​were scaled by a temperature coefficient τ=0.07 and then normalized to a probability distribution using the Softmax function:

[0062] The temperature coefficient τ = 0.07 is a standard configuration in CLIP pre-training. This temperature value, less than 1, acts to "sharpen" the probability distribution—making high-similarity tags have higher probability values ​​and low-similarity tags have lower probability values, thereby amplifying the semantic discrimination signal. This yields the semantic attribute probability distribution vector. Each of its components satisfies and .

[0063] in, The probability that the image to be identified belongs to the k-th semantic label is represented; sim(·,·) represents the cosine similarity between two vectors; This represents the mapped visual features after mapping to the text feature space; This represents the text feature vector corresponding to the k-th semantic label; τ represents the text feature vector corresponding to the j-th semantic label; τ represents the temperature coefficient; k and j are both label indices, with values ​​ranging from 1 to 32, i.e., traversing all 32 semantic labels.

[0064] For example, if an image shows a person wearing a red hard hat, blue overalls, of medium build, and carrying a tool bag, then in the 32-dimensional probability distribution, the dimensions corresponding to "wearing a red hard hat" and "wearing blue overalls" will receive higher probability values ​​(e.g., 0.35 and 0.28), while the probability values ​​for dimensions corresponding to irrelevant labels such as "wearing a white hard hat" and "wearing orange overalls" will approach 0. Even if two people are wearing identical overalls (blue overalls), the differences in their hard hat color (red vs. yellow) and the items they carry (tool bag vs. empty-handed) will still create distinguishable patterns in the probability distribution.

[0065] The original 32-dimensional semantic probability distribution p has sparsity and a probabilistic simplex constraint (the sum of each component is 1). When directly used as a semantic embedding and concatenated with visual features, its numerical range (0,1) differs significantly from that of visual features (which typically have larger amplitudes). Furthermore, the semantic representation capacity of 32 dimensions is limited and insufficient for equal integration with 512-dimensional visual features. Therefore, this embodiment uses a two-layer perceptron (MLP) to map the 32-dimensional probability distribution to a 128-dimensional dense semantic embedding space.

[0066] The structure of this MLP is as follows:

[0067] Specifically: First layer (32 → 256): , Followed by the ReLU activation function: And add Dropout regularization (dropout rate 0.2) to prevent overfitting; Second layer (256 → 128): , No activation function (linear output).

[0068] Where p represents the probability distribution of the semantic attribute, and its dimension is 32; This represents the weight matrix of the first layer of the MLP, which has a dimension of 256×32; This represents the bias vector of the first layer of the MLP, which has a dimension of 256. This represents the weight matrix of the second layer of the MLP, which has a dimension of 128×256; represents the bias vector of the second layer of the MLP, with a dimension of 128; ReLU(·) represents the linear rectified activation function; Dropout(·) represents the random deactivation regularization operation; s represents the output semantic embedding vector, with a dimension of 128.

[0069] The semantic embedding vector is obtained after MLP mapping. : s=MLP(p).

[0070] The semantic embedding vector s is concatenated with the 512-dimensional multi-granular visual features output by the subsequent foreground decoupling module to form a final 640-dimensional identity embedding vector. This is used for final identity retrieval and matching.

[0071] The parameters of the MLP During training, the adapter parameters are updated along with the training process. The initialization uses He normal initialization to ensure that the variance of the activation values ​​remains stable during forward propagation. The introduction of MLP enables the 32-dimensional semantic probability signal to obtain nonlinear transformation and dimensional expansion capabilities (32→128), avoiding the suppression of the semantic signal in terms of numerical amplitude by the 512-dimensional visual features, and ensuring that semantic information plays a substantial discriminative role in the final identity embedding.

[0072] To ensure that the cross-modal similarity calculation accurately reflects the correspondence between visual features and semantic labels, this embodiment introduces cross-modal alignment loss during the training phase. Using the InfoNCE comparative learning approach:

[0073] in, Indicates cross-modal alignment loss; Represents the mapped visual features of the i-th training sample within the batch; This represents the average value of the text feature vectors corresponding to all positive labels in the training sample; represents the text feature vector corresponding to the j-th semantic tag; sim(·,·) represents the cosine similarity; τ represents the temperature coefficient, which is taken as 0.07 in this embodiment; j is the tag index, with a value range of 1 to 32.

[0074] in, This is the average of the text feature vectors of all positive labels (i.e., the semantic attributes that the image actually corresponds to) for the current image. Identity Contrast Loss Ensure that the features of the same person are consistent across different cameras:

[0075] Features of positive samples with the same identity (different cameras / time periods). This refers to all samples within the batch. The temperature coefficient representing the identity comparison loss is used to adjust the sharpness of the identity similarity distribution. In this embodiment, τ′=0.05.

[0076] in, This indicates a loss in identity comparison; This represents the final identity embedding vector of the i-th training sample within the batch; This represents the final identity embedding vector corresponding to a positive sample that belongs to the same identity as the i-th training sample but comes from a different camera or a different time period; represents the final identity embedding vector of the j-th training sample in the batch; sim(·,·) represents the cosine similarity; The temperature coefficient represents the identity comparison loss; j is the sample index, which iterates through all samples in the current batch.

[0077] This loss function encourages visual features. In the joint embedding space, it converges to the center of text features with all positive labels while moving away from text features with negative labels. During training, this loss is compared with the identity contrast loss. and foreground segmentation auxiliary loss Weighted joint optimization enables the visual encoder and adapter to gradually adapt to the discrimination requirements of specific personnel attribute distributions in industrial scenarios while retaining CLIP's general semantic understanding capabilities. Among these, the total loss... = +0.5× +0.1× .

[0078] During the inference phase, the aforementioned text feature matrix T and projection layer... Temperature coefficient Both MLP and MLP remain fixed, only performing forward propagation computation, without the need for a loss function.

[0079] In this embodiment, the cross-modal alignment loss, identity contrast loss, and foreground segmentation auxiliary loss are jointly optimized during the training phase. A preferred configuration of their training hyperparameters is as follows: the AdamW optimizer is used, and the learning rate is set to 2e. 4. Weight decay is set to 0.01; cosine annealing is used for learning rate scheduling, with an annealing period T. maxThe learning rate is set to 60, and a warmup is performed during the first 5 epochs of training. A p-sampling strategy is used for training samples, where P=16 identities are randomly selected in each batch, and K=4 images are randomly sampled for each identity, with a batch size of 64. The total training epochs are 60, requiring approximately 8 hours to complete on 4 A100 GPUs. Early stopping is triggered when the Rank-1 accuracy on the validation set no longer improves for 5 consecutive epochs, and the model parameters with the highest mean average precision (mAP) on the validation set are selected as the final model. Those skilled in the art can adjust the above hyperparameters according to the training data scale and hardware conditions, which does not constitute a limitation on the scope of protection of this application.

[0080] Furthermore, in one embodiment, the dimensions and number of semantic tags can be adjusted according to the specific industrial scenario, and are not limited to four dimensions and 32 tags; the text description template can adopt various natural language expressions, and is not limited to the fixed sentence structure "A person in industrial site, [description]"; the temperature coefficient τ can be adjusted within the range of 0.05 to 0.10 to control the sharpness of the probability distribution; the mapping from the semantic probability distribution to the semantic embedding is not limited to two-layer MLP, but can also adopt a single-layer linear transformation or three or more layers of MLP, or use a Transformer encoder for more complex semantic modeling; when there is a lack of pre-trained text encoders such as CLIP, an independent attribute recognition network can be used to replace the text encoder to directly predict the attribute tag probability, or a knowledge graph embedding can be used to replace the text template library to model the relationship between attributes through graph relations.

[0081] Step S30: Based on the visual encoder, the visual feature vector, and the semantic embedding vector, the final identity embedding vector is obtained through foreground saliency decoupling. In some specific embodiments, step S30 includes: Step S301: Generate a saliency map based on the attention weights inside the visual encoder; In some specific embodiments, step S301 includes: Extract the attention weights of the classification tokens to each image patch token in the last layer of the visual encoder; The results of each attention head are averaged, and the averaged attention weights are mapped back to the spatial dimensions of the image to be identified to obtain a saliency map.

[0082] In this embodiment, background elements such as dust, light spots, and moving machinery in industrial scenes introduce significant noise during feature extraction. This invention designs a foreground-background saliency decoupling mechanism to separate the human foreground from background interference at the source of feature extraction. This mechanism comprises four steps: saliency map generation, foreground attention constraint, foreground weighted feature aggregation, and multi-granularity key part representation.

[0083] Saliency map generation: From the multi-head self-attention matrix of the 12th (last) Transformer block of ViT, extract the attention weights of the CLS token to the 128 Patch tokens. Each of the 12 attention heads generates a set of weight vectors. Take the element-wise average of the 12 heads to get .Will Reconstructed into a 16×8 two-dimensional saliency map After bilinear interpolation and upsampling to 256×128, the result is... . Each element reflects the contribution of the corresponding pixel to the global representation of CLS, and the value range is normalized to [0,1] by softmax. Ideally, the attention weight of the human body region should be higher than that of the background, but under dust / reflective conditions, the original attention of ViT is often attracted by high-contrast background noise, so foreground constraints are needed.

[0084] in, This represents the attention weight vector output by the h-th attention head in the last layer of the visual encoder, with a dimension of 128 and a value of h ranging from 1 to 12. M represents the element-wise average of the attention weight vectors of the 12 attention heads; M represents the result of... The reconstructed two-dimensional saliency map has a size of 16×8. This represents the significance graph after bilinear interpolation upsampling to 256×128.

[0085] Furthermore, in one embodiment, if a CNN is used as the feature extraction backbone (such as ResNet or HRNet), the saliency map can be generated using GradCAM (gradient-weighted class activation mapping) or Grad-CAM++ methods. By performing global average pooling on the gradient information of the target layer, a saliency heatmap of the same size is generated, which is equivalent to the ViT attention saliency map described in this embodiment. In addition to the last layer, the penultimate layer or the attention weights of multiple intermediate layers can also be extracted and weighted and fused to balance the saliency expression of low-level texture information and high-level semantic information. In addition to element-wise averaging, other aggregation methods such as weighted averaging (assigning learnable fusion weights to different attention heads) or max pooling can also be used. In addition to bilinear interpolation, nearest neighbor interpolation, bicubic interpolation, or learning-based super-resolution upsampling modules can also be used to achieve the accuracy requirements of pixel-level saliency maps.

[0086] Step S302: Correct the saliency map using foreground segmentation constraints to obtain a foreground saliency map; In some specific embodiments, step S302 includes: The feature map output by the intermediate layer of the visual encoder is received by a lightweight auxiliary network to obtain the foreground probability map output by the auxiliary network; wherein, the auxiliary network is trained in advance using a semantic segmentation mask automatically generated by a pre-trained human parsing model as a supervision signal. The foreground probability map is multiplied element-wise with the saliency map to obtain the foreground saliency map.

[0087] In this embodiment, foreground attention is constrained by introducing a lightweight foreground segmentation auxiliary head to constrain the saliency map. The auxiliary head structure is a 3-layer convolutional network: Conv(768→64, 3×3, pad=1) → BN → ReLU → Conv(64→32, 3×3, pad=1) → BN → ReLU → Conv(32→1, 1×1) → Sigmoid, with approximately 160,000 parameters. The auxiliary head receives the patch features from the 6th layer of ViT (an intermediate layer that combines low-level texture and mid-level semantic information) as input, reshapes it into a 16×8×768 feature map, and outputs a foreground probability map after convolution. During training, the supervision signal is automatically generated by the pre-trained HRNet-W48 human parsing model through inference on the training images (no additional manual annotation is required), and the loss is binary cross-entropy. During reasoning, Upsampled to 256×128 and then Element-wise multiplication yields the foreground enhancement saliency plot: = ⊙

[0088] Where ⊙ represents the Hadamard product. The result is: Attention was diverted from background areas such as dust, light spots, and machinery. Low-value suppression preserves human attention. Auxiliary head inference time is <1ms, having no impact on overall pipeline performance.

[0089] in, Represents foreground saliency; This represents the saliency map generated by the attention weights; The foreground probability map output by the auxiliary network has a size of 16×8. This represents the foreground probability map after upsampling to 256×128; ⊙ represents the element-wise multiplication (Hadamard product) operation.

[0090] That is, the high response value of the human foreground region in the foreground probability map is used to retain the attention weight of the corresponding region in the saliency map, while the low response value of the background region in the foreground probability map is used to suppress the attention weight of the corresponding region in the saliency map.

[0091] Furthermore, in one embodiment, the auxiliary network is not limited to 3 convolutional layers, but may also employ deeper stacked convolutional layers (such as 5 layers) or lightweight structures based on depthwise separable convolutions (such as MobileNet-style inverted residual blocks) to achieve different balances between parameter quantity and segmentation accuracy. Pre-trained human parsing models are not limited to HRNet-W48. More efficient lightweight segmentation models (such as BiSeNet, FPN, etc.) can be used to generate pseudo-labels, or the recently released Segment Anything Model (SAM) can be used to automatically generate high-quality foreground masks. The auxiliary network is not limited to receiving features from the 6th layer; it can also receive features from the 4th layer (which emphasizes low-level texture) or the 8th layer (which emphasizes mid-level semantics), or input features from multiple intermediate layers after weighted fusion. Foreground probability maps and saliency maps are not limited to element-wise multiplication (Hadamard product). Alternative strategies such as element-wise addition followed by normalization, gating mechanisms, or attention residual connections can also be used to achieve background suppression. The auxiliary network can be jointly trained end-to-end with the main network, rather than being frozen after pre-training, so that the foreground prediction ability of the auxiliary network can be better adapted to the feature distribution of the main network.

[0092] Step S303: Use the foreground saliency map to perform weighted aggregation on the visual feature vector to obtain global features for foreground enhancement; In this embodiment, foreground weighted feature aggregation is achieved by utilizing... We perform weighted aggregation on the 128 patch token features of ViT layer 12, replacing the traditional CLS token direct output as the global representation:

[0093] For the first Each patch feature The weights correspond to the saliency of the foreground features. Denominator normalization ensures stable output amplitude. This weighted aggregation optimizes global features. It mainly consists of human body region information. In actual tests under dust conditions, the foreground-weighted aggregation improved the Rank-1 of CLS direct output by 12.3 percentage points.

[0094] in, The global feature representing foreground enhancement has a dimension of 768; This represents the feature vector of the i-th image block token output from the last layer of the visual encoder; The foreground saliency weight corresponding to the i-th image block in the foreground saliency map is represented; i is the image block index, with a value range of 1 to 128.

[0095] Furthermore, in one embodiment, in addition to weighted mean, weighted max pooling can also be used—taking only the... The K largest image patch features are averaged and aggregated to further highlight the most discriminative local regions; Weights can be adjusted Apply temperature scaling ,in, The temperature coefficient is then represented, and aggregation is performed to adjust the "sharpness" of the weight distribution—when When the size is smaller, aggregation is more concentrated in high-weight regions; when When the size is larger, the polymerization is smoother; A lightweight attention network can be designed to... The Patch features are subjected to learnable attention pooling as the initial query, while the output dimension remains unchanged at 768, making the aggregation process more adaptive. Weighted aggregation can be iterated in multiple rounds—the first aggregation yields... Then, calculate the features of each patch and... The similarity, and the similarity with After multiplication, the data is weighted and aggregated again to achieve dual guidance based on "foreground priors + feature similarity".

[0096] Step S304: Divide the foreground region of the image to be identified into multiple sub-regions according to the human body structure, and perform weighted aggregation on the visual features of each sub-region to extract the local features of each sub-region. Step S305: The global feature is fused with each of the local features to obtain multi-granularity visual features; Step S306: Perform cross-modal fusion of the multi-granularity visual features and the semantic embedding vector to generate the final identity embedding vector for identity determination.

[0097] In this embodiment, multi-granularity key part characterization involves further extracting fine-grained features of key human body parts within the foreground region. The 16 rows are divided into three vertical lines: rows 1-5 are the head area (helmet / face), rows 6-11 are the torso area (work clothes / reflective strips), and rows 12-16 are the lower limb area (trousers / boots), each represented by a sub-area. The values ​​are obtained by weighted aggregation of the corresponding Patch features. , , The four-level features are concatenated and then dimensionality reduced by a fully connected layer:

[0098] in, This represents the fused multi-granularity visual features, with a dimension of 512. A global feature indicating foreground enhancement; , , The three sub-regions, head, torso, and lower limbs, represent local features, each with a dimension of 768; [·; ·] represents vector concatenation operation; FC(·) represents a fully connected layer that reduces the 3072-dimensional concatenated features to 512 dimensions.

[0099] The fully connected (FC) layer uses BatchNorm and Dropout(0.3) regularization. The resulting 512-dimensional multi-granular visual features are used for identity matching, and are concatenated with the 128-dimensional semantic embedding MLP(p) to obtain the final 640-dimensional identity embedding vector f. The significance of the multi-granular design: Capture the overall outline and posture. Capture the color of the safety helmet (job level clue). Capture the color of the work clothes and reflective stripes (job type clues). Capturing trousers, shoes, and boots (personalized clothing cues), these four elements complement each other to form a robust identity description.

[0100] Furthermore, in one embodiment, the ratio is not limited to a fixed 5:6:5 line ratio. The division ratio can be adaptively adjusted according to the shooting angle of the people in the specific scene (overhead, low-angle, and level shooting). Alternatively, dynamic division based on human key point detection (such as OpenPose) can be adopted to determine the region boundary based on the detected shoulder, waist, and knee key point positions. It is not limited to the three regions of head / trunk / lower limbs, but can be further subdivided into more regions (such as five regions: head, upper limbs, trunk, lower limbs, and feet), or merged into fewer regions (such as only the head and body). The fusion of four-level features is not limited to simple concatenation followed by a fully connected layer. Attention mechanisms (such as cross attention) can be used to achieve interaction and reweighting between features, or gating mechanisms can be used to adaptively adjust the contribution weights of each feature. The fusion of visual features and semantic embeddings is not limited to direct splicing. Cross-modal attention can be used to allow visual features to be adaptively adjusted according to semantic information, or low-rank fusion can be used to reduce the dimensionality of the spliced ​​features. Before splicing, L2 normalization can be performed on each feature vector to make the numerical range of each modality feature consistent, and to prevent a certain modality from dominating the similarity calculation due to excessively large amplitude.

[0101] Step S40: Based on the final identity embedding vector, perform a similarity search in the preset identity feature library, and determine the recognition result based on the search results.

[0102] In some specific embodiments, step S40 includes: A coarse search is performed using an approximate nearest neighbor index structure to obtain a set of candidate identities from a pre-defined identity feature database; Based on each identity in the candidate identity set, the similarity between its corresponding identity embedding vector and the final identity embedding vector is calculated. The candidate identity set is then filtered and sorted in combination with spatiotemporal constraints, and the identity with the highest matching degree is output as the recognition result. The spatiotemporal constraints include: Based on the time difference between two captures of the person to be identified by different cameras and the spatial distance between the corresponding cameras, it is determined whether the time difference is within a reasonable time window determined by the range of the person's movement speed and the path detour coefficient; In addition, the system verifies whether the movement path of the person to be identified between the cameras is physically reachable based on a pre-built camera topology map.

[0103] In this embodiment, the system performs cross-camera identity matching based on a 640-dimensional enhanced identity embedding vector. The matching employs a two-stage retrieval strategy.

[0104] Phase 1 – Coarse Search: Approximate nearest neighbor search is performed using the FAISS vector index library. Index type: IVF-PQ (Inverted Index + Product Quantization), cluster centers nlist=256, sub-vectors m=8 (each 80-dimensional), quantization precision 8-bit. Query vector q ∈ R 640 Search within the 16 nearest inverted lists (nprobe=16) and return the Top-20 candidates. Retrieval complexity... With a database of 10,000 identities, a single search takes approximately 2ms.

[0105] In the retrieval complexity expression, N represents the total number of registered identity feature vectors in the identity feature database; nlist represents the number of cluster centers in the inverted index, i.e., the number of inverted lists, in this embodiment nlist=256; D represents the dimension of the identity embedding vector, in this embodiment D=640; nprobe represents the number of the most recent inverted lists searched in a single query, in this embodiment nprobe=16; PQ cost This represents the computational cost of calculating the approximate distance between the query vector and a single candidate vector based on product quantization encoding. Since this approximate distance is obtained by performing m lookups and summations from a pre-computed distance lookup table, PQ cost The computational cost is proportional to the number of sub-vectors m, which is 8 in this embodiment; nlist×D represents the computational cost of comparing the query vector with all cluster centers, nprobe×N / nlist×PQ. cost This represents the computational cost of calculating the approximate distance to candidate vectors within the searched inverted list.

[0106] Phase Two—Fine Matching: Perform 640-dimensional full-precision cosine similarity calculation (<0.1ms) and spatiotemporal constraint verification on the Top-20 candidates. Time Constraint: The time difference between two detections must be within [...]. , ]Inside, , , 1.5 is the detour coefficient. Spatial constraints: Physical reachability is verified using a pre-built camera topology graph (adjacency list, including distance and connectivity). Constraints (temporal and spatial) are satisfied and cosine similarity > 1.5. The candidate with the highest similarity score of 0.65 is confirmed as the matched identity; if the highest similarity score is less than 0.65, it is registered as a new identity.

[0107] in, , These represent the lower and upper limits of the reasonable time window, respectively. This indicates the spatial distance between the two cameras; , These represent the lower and upper limits of the personnel's movement speed, respectively, which are taken as 1 m / s and 5 m / s in this embodiment; 1.5 represents the path detour coefficient; The cosine similarity threshold for confirming the same identity is 0.65 in this embodiment.

[0108] Furthermore, the global identity index maintains a feature template for each active identity, retaining the embedding vectors from the 10 most recent observations. Template updates employ an exponential moving average:

[0109] in, This represents the updated identity feature template; This represents the identity feature template before the update. This represents the final identity embedding vector obtained from this observation; the coefficient 0.2 is the template update coefficient β, and the coefficient 0.8 is 1. β.

[0110] The template is designed to adapt to slow changes in appearance (removing a helmet, putting on or taking off a vest) while maintaining stable identity memory. Identity entries not appearing in 50 consecutive observations are marked as "off-field" and removed from the active index (retained in the history database for 30 days). When a new identity is registered, FAISS automatically and incrementally updates the affected IVF buckets, taking <100ms.

[0111] Further, in one embodiment, after step S40, the following is included: Based on the identification results, query the permission whitelist corresponding to the security zones divided by the electronic fence system, determine whether the identity corresponding to the identification results has the permission to enter the current area, and detect abnormal behavior patterns based on the spatiotemporal trajectory of the identity corresponding to the identification results under multiple cameras. If not, an out-of-bounds alarm is generated, and the out-of-bounds alarm or the abnormal behavior pattern is pushed to the security management platform. The abnormal behavior patterns include prolonged stays in non-work areas, repeated entry and exit from restricted areas, reverse movement in one-way passages, and exceeding the maximum allowed working time in designated areas.

[0112] In this embodiment, based on the cross-camera identity association results, the system reconstructs the complete spatiotemporal trajectory of personnel in the mining area, and combines electronic fences to perform boundary crossing detection and behavior analysis.

[0113] The electronic fence divides the mining area into several security zones, each with a defined access whitelist. The permissions of the whitelist are stored in a Redis hash table as a four-tuple of <identity ID, zone ID, valid time period, permission type> (O(1) query). The permission types include "permanent access", "temporary authorization (with validity period)" and "prohibited entry". If the identified person's identity is not in the current zone's whitelist, it is considered an out-of-bounds violation. The end-to-end delay of out-of-bounds detection is <3 seconds, and the link is broken down as follows: image acquisition 100ms → feature extraction 200ms → identity matching 500ms → permission query 100ms → alarm generation 100ms, with the remainder being communication overhead.

[0114] Trajectory behavior analysis detects four abnormal patterns: (a) Prolonged stay: Staying in a non-work area for more than 15 minutes (detected by the same camera continuously and the displacement is less than 10 meters), alarm yellow.

[0115] (b) Abnormal loitering: repeatedly entering and exiting within 50 meters of the restricted area boundary more than 3 times (counting the number of cycles of the boundary camera appearing-disappearing-appearing), alarm orange.

[0116] (c) Reverse movement: moving in the opposite direction in a one-way channel, using a pre-calibrated travel direction vector. With the direction vector of motion Cosine value determination: cos( , A value less than -0.5 indicates reverse movement. The direction of motion is estimated by the displacement of the center of the detection box in two consecutive frames. An orange warning is issued.

[0117] in, This represents the pre-defined one-way channel travel direction vector; The vector represents the direction of movement of the person, which is estimated from the displacement of the center of the detection box in two consecutive frames; cos(·,·) represents the cosine value of the angle between the two vectors.

[0118] (d) Timeout: If the stay in the designated area exceeds the maximum allowable working time (e.g., 30 minutes in the blasting area, 60 minutes in the high-temperature area, as defined by the safety procedure and stored in the configuration library), a red alarm will be triggered.

[0119] The alarm information includes personnel identity (name / employee ID / department, linked to the personnel database via ID), 30-minute trajectory playback ([timestamp, camera ID, screenshot] sequence), and the clearest keyframe from each camera (the highest frame is selected based on detection confidence), forming a traceable chain of evidence. WebSocket pushes data to the security management platform and mobile app in real time, with a latency of <500ms.

[0120] In this embodiment, based on the image of the target object to be identified, a visual feature vector of the image to be identified is obtained through a visual encoder; based on the visual feature vector and multiple predefined semantic tags, cross-modal similarity calculation is performed to obtain a semantic attribute probability distribution, and the semantic attribute probability distribution is mapped to a semantic embedding vector; based on the visual encoder, the visual feature vector, and the semantic embedding vector, a final identity embedding vector is obtained through foreground saliency decoupling; based on the final identity embedding vector, a similarity search is performed in a preset identity feature library, and the identification result is determined according to the search results.

[0121] The introduction of semantic tags provides a discrimination signal independent of appearance, opening up a new discrimination dimension for homogenized appearances; By decoupling from foreground saliency, a complementary description of "overall outline + key parts" is achieved, ensuring the robustness of identity representation; By performing similarity searches in a pre-set identity feature database, high-precision, real-time cross-camera identity matching and security management can be achieved.

[0122] The above steps work together to form a low-cost, high-precision, robust, and deployable complete industrial personnel re-identification solution, which solves the technical problem of low accuracy in cross-camera personnel re-identification in complex industrial scenarios with highly homogeneous appearances, which exists in existing technologies.

[0123] Reference Figure 2 , Figure 2 This is a schematic diagram illustrating the collaborative architecture of the functional modules in the personnel re-identification method provided in this application embodiment. For example... Figure 2 As shown, the aforementioned personnel re-identification method is implemented by five collaborative functional modules: Module 1 is a visual encoding module with an embedded lightweight adapter, used to encode the 256×128 image to be identified obtained by cropping and scaling the personnel detection box by 10% and outputting a visual feature vector; Module 2 is a cross-modal semantic alignment module, used to calculate the semantic attribute probability distribution based on the text features of 32 predefined semantic labels and map it into a 128-dimensional semantic embedding vector; Module 3 is a foreground saliency decoupling module, used to generate a foreground saliency map based on the attention weights of the last layer of the visual encoder and the features of the intermediate layer, suppressing background interference; Module 4 is a multi-granularity feature aggregation module, used to fuse the global features of foreground enhancement with the local features of the three sub-regions of head, torso, and lower limbs, and then concatenate it with the semantic embedding vector to obtain the final identity embedding vector; Module 5 is a retrieval matching and boundary detection module, used to perform a two-stage similarity retrieval in the identity feature database and output the recognition results and alarm information in combination with the permission whitelist.

[0124] in, Figure 2 The dashed boxes in the diagram represent the external input and output resources used in the implementation of the method: the personnel detection box is provided by the front-end personnel detection stage, and after being expanded by 10% and cropped to a size of 256×128, it is sent to module 1; the text template library stores the above 32 semantic tags and their fixed text description templates for module 2 to call; the identity feature library stores the feature templates of registered identities, providing comparison samples for the two-stage similarity retrieval of module 5; the electronic fence and area permission whitelist store the access permission information of each security zone, for module 5 to perform area permission verification; module 5 finally outputs the recognition results and boundary crossing alarms to the security management platform.

[0125] Based on the same inventive concept, embodiments of this application also provide a person re-identification device.

[0126] In one embodiment, reference is made to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of the personnel re-identification device of this application. Figure 3 As shown, the personnel re-identification device includes: The visual encoding module 10 is configured to obtain the visual feature vector of the image to be identified based on the image of the target object through a visual encoder. The semantic alignment module 20 is configured to perform cross-modal similarity calculation based on the visual feature vector and multiple predefined semantic labels, obtain a semantic attribute probability distribution, and map the semantic attribute probability distribution into a semantic embedding vector; The foreground decoupling module 30 is configured to obtain the final identity embedding vector based on the visual encoder, the visual feature vector, and the semantic embedding vector through foreground saliency decoupling. The retrieval and matching module 40 is configured to perform a similarity search in a preset identity feature library based on the final identity embedding vector, and determine the recognition result based on the search results.

[0127] Optionally, in one embodiment, the visual encoding module 10 is configured to: employ a pre-trained feature extraction backbone network, wherein a lightweight adapter module is embedded in multiple network layers of the backbone network; The adapter module includes a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, and adds the output of the adapter module to the original output of the network layer through a residual connection; During training, the original parameters of the backbone network are frozen, and only the internal parameters of the adapter module and the learnable scaling factor of the adapter module are updated.

[0128] Optionally, in one embodiment, the semantic alignment module 20 is configured to: Obtain the text description corresponding to each semantic tag; The text descriptions corresponding to each semantic tag are encoded into corresponding text feature vectors by a frozen text encoder, forming a set of text feature vectors; The visual feature vectors are mapped to the space containing the text feature vector set to obtain the mapped visual features; Calculate the similarity between the mapped visual features and each text feature vector in the text feature vector set; The probability distribution of the image to be identified on each semantic label is generated based on the similarity, and used as the probability distribution of semantic attributes.

[0129] Optionally, in one embodiment, the foreground decoupling module 30 is configured to: A saliency map is generated based on the attention weights within the visual encoder; The saliency map is corrected using foreground segmentation constraints to obtain a foreground saliency map; The visual feature vectors are weighted and aggregated using the foreground saliency map to obtain global features for foreground enhancement; The foreground region of the image to be identified is divided into multiple sub-regions based on the human body structure. The visual features within each sub-region are weighted and aggregated to extract the local features of each sub-region. The global features are fused with each of the local features to obtain multi-granularity visual features; The multi-granular visual features are fused with the semantic embedding vector across modalities to generate the final identity embedding vector for identity determination.

[0130] Optionally, in one embodiment, the foreground decoupling module 30 is configured to: Extract the attention weights of the classification tokens to each image patch token in the last layer of the visual encoder; The results of each attention head are averaged, and the averaged attention weights are mapped back to the spatial dimensions of the image to be identified to obtain a saliency map.

[0131] Optionally, in one embodiment, the foreground decoupling module 30 is configured to: The feature map output by the intermediate layer of the visual encoder is received by a lightweight auxiliary network to obtain the foreground probability map output by the auxiliary network; wherein, the auxiliary network is trained in advance using a semantic segmentation mask automatically generated by a pre-trained human parsing model as a supervision signal. The foreground probability map is multiplied element-wise with the saliency map to obtain the foreground saliency map.

[0132] Optionally, in one embodiment, the retrieval matching module 40 is configured to: A coarse search is performed using an approximate nearest neighbor index structure to obtain a set of candidate identities from a pre-defined identity feature database; Based on each identity in the candidate identity set, the similarity between its corresponding identity embedding vector and the final identity embedding vector is calculated. The candidate identity set is then filtered and sorted in combination with spatiotemporal constraints, and the identity with the highest matching degree is output as the recognition result. The spatiotemporal constraints include: Based on the time difference between two captures of the person to be identified by different cameras and the spatial distance between the corresponding cameras, it is determined whether the time difference is within a reasonable time window determined by the range of the person's movement speed and the path detour coefficient; In addition, the system verifies whether the movement path of the person to be identified between the cameras is physically reachable based on a pre-built camera topology map.

[0133] Optionally, in one embodiment, the personnel re-identification device further includes a boundary crossing alarm module 50, configured to: Based on the identification results, query the permission whitelist corresponding to the security zones divided by the electronic fence system, determine whether the identity corresponding to the identification results has the permission to enter the current area, and detect abnormal behavior patterns based on the spatiotemporal trajectory of the identity corresponding to the identification results under multiple cameras. If not, an out-of-bounds alarm is generated, and the out-of-bounds alarm or the abnormal behavior pattern is pushed to the security management platform. The abnormal behavior patterns include prolonged stays in non-work areas, repeated entry and exit from restricted areas, reverse movement in one-way passages, and exceeding the maximum allowed working time in designated areas.

[0134] The functions of each module in the aforementioned personnel re-identification device correspond to the steps in the aforementioned personnel re-identification method embodiment, and their functions and implementation processes will not be described in detail here.

[0135] Based on the same inventive concept, embodiments of this application also provide an electronic device, the structure of which is as follows: Figure 4 As shown, it includes: a memory and a processor, wherein the processor is used to read and execute the computer program stored in the memory to implement the aforementioned person re-identification method.

[0136] Based on the same inventive concept, this application also provides a computer storage medium storing computer-executable instructions, which, when executed, implement the aforementioned person re-identification method.

[0137] Finally, it should be noted that while some processes described in the embodiments of this application include multiple operations or steps that appear in a specific order, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0138] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for re-identifying persons, characterized in that, The method includes: Based on the image to be identified of the target object, a visual feature vector of the image to be identified is obtained through a visual encoder; Based on the visual feature vector and multiple predefined semantic labels, cross-modal similarity calculation is performed to obtain the semantic attribute probability distribution, and the semantic attribute probability distribution is mapped to a semantic embedding vector; Based on the visual encoder, the visual feature vector, and the semantic embedding vector, the final identity embedding vector is obtained through foreground saliency decoupling. Based on the final identity embedding vector, a similarity search is performed in a preset identity feature database, and the recognition result is determined according to the search results.

2. The person re-identification method according to claim 1, characterized in that, The visual encoder employs a pre-trained feature extraction backbone network, and lightweight adapter modules are embedded in multiple network layers of the backbone network. The adapter module includes a dimension reduction projection layer, a nonlinear activation layer, and a dimension increase projection layer, and adds the output of the adapter module to the original output of the network layer through a residual connection; During training, the original parameters of the backbone network are frozen, and only the internal parameters of the adapter module and the learnable scaling factor of the adapter module are updated.

3. The person re-identification method according to claim 1, characterized in that, The step of calculating cross-modal similarity based on the visual feature vector and multiple predefined semantic labels to obtain the semantic attribute probability distribution includes: Obtain the text description corresponding to each semantic tag; The text descriptions corresponding to each semantic tag are encoded into corresponding text feature vectors by a frozen text encoder, forming a set of text feature vectors; The visual feature vectors are mapped to the space containing the text feature vector set to obtain the mapped visual features; Calculate the similarity between the mapped visual features and each text feature vector in the text feature vector set; The probability distribution of the image to be identified on each semantic label is generated based on the similarity, and used as the probability distribution of semantic attributes.

4. The person re-identification method according to claim 1, characterized in that, The process of obtaining the final identity embedding vector based on the visual encoder, the visual feature vector, and the semantic embedding vector through foreground saliency decoupling includes: A saliency map is generated based on the attention weights within the visual encoder; The saliency map is corrected using foreground segmentation constraints to obtain a foreground saliency map; The visual feature vectors are weighted and aggregated using the foreground saliency map to obtain global features for foreground enhancement; The foreground region of the image to be identified is divided into multiple sub-regions based on the human body structure. The visual features within each sub-region are weighted and aggregated to extract the local features of each sub-region. The global features are fused with each of the local features to obtain multi-granularity visual features; The multi-granular visual features are fused with the semantic embedding vector across modalities to generate the final identity embedding vector for identity determination.

5. The person re-identification method according to claim 4, characterized in that, The generation of a saliency map based on the attention weights within the visual encoder includes: Extract the attention weights of the classification tokens to each image patch token in the last layer of the visual encoder; The results of each attention head are averaged, and the averaged attention weights are mapped back to the spatial dimensions of the image to be identified to obtain a saliency map.

6. The person re-identification method according to claim 4, characterized in that, The step of modifying the saliency map using foreground segmentation constraints to obtain a foreground saliency map includes: The feature map output by the intermediate layer of the visual encoder is received by a lightweight auxiliary network to obtain the foreground probability map output by the auxiliary network; wherein, the auxiliary network is trained in advance using a semantic segmentation mask automatically generated by a pre-trained human parsing model as a supervision signal. The foreground probability map is multiplied element-wise with the saliency map to obtain the foreground saliency map.

7. The person re-identification method according to claim 1, characterized in that, The step of performing a similarity search in a preset identity feature database based on the final identity embedding vector, and determining the recognition result based on the search results, includes: A coarse search is performed using an approximate nearest neighbor index structure to obtain a set of candidate identities from a pre-defined identity feature database; Based on each identity in the candidate identity set, the similarity between its corresponding identity embedding vector and the final identity embedding vector is calculated. The candidate identity set is then filtered and sorted in combination with spatiotemporal constraints, and the identity with the highest matching degree is output as the recognition result. The spatiotemporal constraints include: Based on the time difference between two captures of the person to be identified by different cameras and the spatial distance between the corresponding cameras, it is determined whether the time difference is within a reasonable time window determined by the range of the person's movement speed and the path detour coefficient; In addition, the system verifies whether the movement path of the person to be identified between the cameras is physically reachable based on a pre-built camera topology map.

8. The person re-identification method according to claim 1, characterized in that, After performing a similarity search in a preset identity feature database based on the final identity embedding vector, and determining the recognition result based on the search results, the process includes: Based on the identification results, query the permission whitelist corresponding to the security zones divided by the electronic fence system, determine whether the identity corresponding to the identification results has the permission to enter the current area, and detect abnormal behavior patterns based on the spatiotemporal trajectory of the identity corresponding to the identification results under multiple cameras. If not, an out-of-bounds alarm is generated, and the out-of-bounds alarm or the abnormal behavior pattern is pushed to the security management platform. The abnormal behavior patterns include prolonged stays in non-work areas, repeated entry and exit from restricted areas, reverse movement in one-way passages, and exceeding the maximum allowed working time in designated areas.

9. A person re-identification device, characterized in that, The device includes: The visual encoding module is configured to obtain the visual feature vector of the image to be identified based on the image of the target object through a visual encoder. The semantic alignment module is configured to perform cross-modal similarity calculation based on the visual feature vector and multiple predefined semantic labels, obtain the semantic attribute probability distribution, and map the semantic attribute probability distribution into a semantic embedding vector; The foreground decoupling module is configured to obtain the final identity embedding vector based on the visual encoder, the visual feature vector, and the semantic embedding vector through foreground saliency decoupling. The retrieval and matching module is configured to perform a similarity search in a preset identity feature library based on the final identity embedding vector, and determine the recognition result based on the search results.

10. An electronic device, characterized in that, include: Memory, processor; The processor is configured to read and execute the computer program stored in the memory to implement the steps of the person re-identification method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, implement the steps of the person re-identification method according to any one of claims 1-8.

Images