Cross-modal pedestrian re-identification and training method

Abstract

The invention discloses a cross-modal pedestrian re-identification and training method, and solves the problems of semantic mismatch, insufficient optimization of difficult cases and unbalanced view angle distribution in visible light-infrared pedestrian re-identification (VI-ReID). The method comprises the following steps: ensuring multi-view data equalization by a view equalization sampler (VB); the basic feature extraction network generates a high-dimensional feature map; a loop semantic aggregation module (RSA) is combined with a bidirectional long-short term memory network (BiLSTM) to optimize local and global features; and constructing a modal alignment loss function to enhance the cross-modal feature consistency. Furthermore, the difficult case recognition capability is improved through a confidence-based difficult case retraining strategy (CHSR). Compared with the traditional method, the method provided by the invention has the advantages that the Rank-1 accuracy and the average precision (mAP) are remarkably improved. The method is suitable for a multi-mode pedestrian re-identification scene, and has important application value in the fields of intelligent monitoring and public safety.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep learning, and in particular to a cross-modal pedestrian re-identification and training method. Background Art

[0002] Person Re-Identification (ReID) is an important task in the field of computer vision, which aims to achieve cross-camera pedestrian identity matching by analyzing pedestrian images taken by different cameras. This technology has wide application value in the fields of intelligent monitoring and public safety, such as assisting in building monitoring networks, accurately tracking target personnel, and improving case detection efficiency.

[0003] Traditional ReID methods mainly rely on visible light cameras, which work well in well-lit environments, but their performance drops significantly in low-light or no-light environments (such as at night and in underground parking lots). To address this problem, modern surveillance has gradually introduced infrared cameras, which has promoted the development of visible-infrared person re-identification (VI-ReID). By combining visible light and infrared images, VI-ReID overcomes the limitations of a single modality and achieves pedestrian identity matching in all weather and all scenarios.

[0004] VI-ReID technology is particularly important in public security. For example, it can match pedestrian images in different modes at night or in bad weather conditions to accurately track suspects. It also plays an important role in border monitoring and security in unmanned areas. By improving recognition capabilities in weak light and complex environments, VI-ReID provides technical support for the upgrade of intelligent monitoring systems and promotes the construction of a more efficient and safer social environment.

[0005] VI-ReID aims to match pedestrian images in different modalities (visible light and infrared light) and from different camera perspectives. However, due to the significant differences in modalities, VI-ReID faces many technical challenges. First, existing methods (such as MAUM and MID) mostly extract features by horizontally dividing local regions, but this method is prone to semantic mismatch problems when facing pedestrian posture changes or subtle displacements. For example, the correlation between different local regions may be ignored, resulting in incomplete feature extraction, thereby reducing the model's ability to distinguish different pedestrians. In addition, the cross-modal characteristics of VI-ReID make it difficult for existing models to effectively distinguish between images of different pedestrians with high similarity, and the model's learning of difficult examples is still a problem that needs to be solved. Although the triplet loss and its variants have alleviated this problem to a certain extent, existing methods usually only focus on the selection of difficult examples in small batches of data, while ignoring the optimization of difficult examples in the global scope. This limitation further restricts the robustness and generalization ability of the model.

[0006] Another significant problem is that the existing sampling strategy fails to ensure that data from different camera perspectives are evenly sampled during training. This imbalance causes data from some camera perspectives to be overtrained in the training data, while data from other perspectives is ignored. This data imbalance further weakens the model's ability to adapt to scarce perspectives and affects the overall performance.

[0007] Although the industry has proposed some improvement schemes in recent years, such as the feature extraction method combining global and local features proposed by SAAI, and the difficult example mining strategy based on Triplet Loss, these methods have alleviated the above problems to a certain extent, but there are still obvious limitations. Specifically, although SAAI enhances the feature expression ability by fusing global and local features, it lacks an effective difficult example mining mechanism, resulting in limited improvement in the robustness of the model when dealing with complex samples. On the other hand, Triplet Loss mines difficult examples by constructing triplet samples, but its mining strategy is limited to difficult examples within a small batch (batch), and fails to make full use of difficult examples in the global data set. This limitation significantly weakens the model's ability to distinguish complex samples globally.

[0008] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0009] The main purpose of the present invention is to overcome the defects existing in the above-mentioned background technology and provide a cross-modal pedestrian re-identification and training method.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] A cross-modal person re-identification training method includes the following steps:

[0012] S1. Perform data preprocessing on pedestrian images in visible light and infrared modalities. Use the view balanced sampler (VB) to construct a multi-view balanced mini-batch dataset from each modality by sampling the camera view distribution evenly, so as to balance the data distribution of different camera views.

[0013] S2, input small batches of data into the basic feature extraction network to generate semantically rich high-dimensional feature maps;

[0014] S3. Optimize the high-dimensional feature map using the recurrent semantic aggregation module (RSA), where the high-dimensional feature map is divided into local blocks, and the local blocks are gradually aggregated through the bidirectional long short-term memory network (BiLSTM) to generate a global and local semantic feature representation, and optimize the consistency of cross-modal feature distribution by constructing a modality alignment loss function;

[0015] The training data generated in this way is continuously input into the model, the overall loss function is calculated, and the model parameters are iteratively optimized until the training meets the preset convergence conditions.

[0016] Furthermore, in step S2, the basic feature extraction network adopts a ResNet-50 model.

[0017] Furthermore, in step S3, the recurrent semantic aggregation module (RSA) optimizes the high-dimensional feature graph by specifically performing the following steps:

[0018] S31, dividing the high-dimensional feature map into several local blocks, and extracting local features by average pooling;

[0019] S32, local blocks are aggregated to capture semantic associations through a bidirectional long short-term memory network (BiLSTM), and BiLSTM gradually fuses local semantic associations during sequence processing;

[0020] S33, extracting global features from the entire feature map through global average pooling;

[0021] S34, concatenating local features and global features to form a complete semantic feature representation;

[0022] S35. For the complete semantic feature representation obtained by concatenation, the consistency of cross-modal features is optimized through the modal alignment loss function.

[0023] Furthermore, the method further comprises the following steps:

[0024] S4. Calculate the classification confidence score of the sample, filter out the hard negative samples according to the confidence, retrain the filtered hard samples using the confidence-based hard retraining strategy (CHSR), and optimize the training process of the hard samples based on the group identity loss and the group center separation loss.

[0025] Furthermore, step S4 specifically includes:

[0026] S41, calculating the classification confidence score of the training sample through the classifier, and selecting the negative sample with the highest confidence from the global data set as the difficult example for retraining;

[0027] S42. Use group identity loss and group center separation loss to optimize the relationship between difficult negative samples and original samples.

[0028] Furthermore, during the training process, the group identity loss function, group center separation loss function, modality alignment loss function and difficult example group loss function are used to calculate the model error, and the model parameters are updated according to the error through the back propagation algorithm to optimize the performance of the basic feature extraction network, the cyclic semantic aggregation module and the difficult example optimization module.

[0029] A cross-modal pedestrian re-identification method is disclosed, which realizes cross-modal pedestrian re-identification by using a model trained by the cross-modal pedestrian re-identification training method.

[0030] A computer-readable storage medium stores a computer program, which implements the method when executed by a processor.

[0031] A computer program product comprises a computer program, which implements the method when executed by a processor.

[0032] The present invention has the following beneficial effects:

[0033] The present invention provides a cross-modal pedestrian re-identification and training method, which innovatively solves multiple key problems in the field of visible light-infrared pedestrian re-identification (VI-ReID) and significantly improves the performance and robustness of the model. First, by introducing the model design of the view balanced sampler (VB), the present invention ensures the balanced distribution of data under different camera perspectives, effectively solves the problem of unbalanced view distribution in the existing sampling strategy, enhances the model's adaptability to scarce view angles, and thus improves the overall performance. Secondly, by using the recurrent semantic aggregation module (RSA) and the bidirectional long short-term memory network (BiLSTM), the present invention can capture the global and local features in pedestrian images and optimize the consistency of cross-modal feature distribution, effectively solving the semantic mismatch problem and improving the model's ability to distinguish different pedestrians. Furthermore, the present invention not only focuses on the selection of difficult examples in small batches of data, but also optimizes difficult examples globally through a confidence-based difficult example retraining strategy (CHSR), thereby further improving the robustness and generalization ability of the model. Experimental verification on VI-ReID benchmark datasets such as SYSU-MM01 and RegDB shows that the present invention significantly outperforms existing methods in both Rank-1 accuracy and mean average precision (mAP), achieving the highest performance. The technical solution of the present invention is suitable for multimodal pedestrian re-identification scenarios and can be widely used in fields such as intelligent monitoring and public safety, providing strong technical support for building an all-weather, all-scenario pedestrian identity matching system.

[0034] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of a training method for a semantic alignment and hard example optimization method for cross-modal person re-identification according to an embodiment of the present invention;

[0036] Figure 2 It is a model schematic diagram of the semantic alignment and hard-example optimization method for cross-modal person re-identification according to an embodiment of the present invention.

[0037] Figure 3 It is a schematic diagram of the structure of the basic feature extraction network ResNet-50 of an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0039] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for fixing as well as for coupling or communication.

[0040] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0041] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0042] In terms of achieving effective pedestrian identity matching between visible light and infrared modalities, the present invention addresses the common challenges in the field of visible light-infrared pedestrian re-identification (VI-ReID), including semantic mismatch, unbalanced view distribution and insufficient optimization of difficult examples, and proposes an innovative cross-modal pedestrian re-identification training method and a cross-modal pedestrian re-identification method based on this.

[0043] See also Figure 1 and Figure 2 , an embodiment of the present invention provides a training method for cross-modal person re-identification, comprising the following steps:

[0044] Step S1, data preprocessing and perspective balanced sampling: Data preprocessing is performed on pedestrian images in visible light and infrared modes, and a perspective balanced sampler (VB) is used to construct a multi-perspective balanced mini-batch data set from each mode to balance the data distribution of different camera perspectives. The perspective balanced sampler (VB) ensures that the training batch contains multi-perspective data by sampling the camera perspective distribution evenly, reducing the model's sensitivity to scarce perspectives.

[0045] Step S2, high-dimensional feature map extraction: input small batch data into the basic feature extraction network to generate semantically rich high-dimensional feature maps.

[0046] Preferably, the basic feature extraction network adopts a ResNet-50 model, which extracts high-dimensional feature maps through its deep convolutional structure, providing semantically rich feature input for subsequent modules.

[0047] Step S3, recurrent semantic aggregation: using a recurrent semantic aggregation module (RSA) to optimize the high-dimensional feature map, wherein the high-dimensional feature map is divided into local blocks, and the local blocks are gradually aggregated through a bidirectional long short-term memory network (BiLSTM) to generate a global and local semantic feature representation, and the consistency of cross-modal feature distribution is optimized by constructing a modal alignment loss function;

[0048] In a preferred embodiment, the recurrent semantic aggregation module (RSA) optimizes the high-dimensional feature graph by the following steps:

[0049] Step S31, dividing the high-dimensional feature map into several local blocks, and extracting local features by average pooling;

[0050] Step S32, aggregating local blocks through a bidirectional long short-term memory network (BiLSTM) to capture semantic associations, and BiLSTM gradually integrates local semantic associations during sequence processing;

[0051] Step S33, extracting global features from the entire feature map by global average pooling;

[0052] Step S34, concatenating local features and global features to form a complete semantic feature representation;

[0053] Step S35: For the complete semantic feature representation obtained by concatenation, the consistency of cross-modal features is optimized through the modality alignment loss function.

[0054] In a preferred embodiment, the cross-modal person re-identification training method further comprises the following steps:

[0055] Step S4, hard example retraining: calculate the classification confidence score of the sample, screen the hard negative samples according to the confidence, use the confidence-based hard example retraining strategy (CHSR) to retrain the screened hard samples, and optimize the training process of the hard samples according to the group identity loss and the group center separation loss.

[0056] Specifically, it includes: S41, calculating the classification confidence score of the training sample through the classifier, and selecting the negative samples with the highest confidence from the global data set as difficult examples for retraining; S42, optimizing the relationship between the difficult negative samples and the original samples by using the group identity loss and the group center separation loss.

[0057] During the training process, the training data generated by steps S1 to S4 are continuously input into the model, the overall loss function is calculated, and the model parameters are iteratively optimized until the training meets the preset convergence conditions. Specifically, the group identity loss function, the group center separation loss function, the modality alignment loss function, and the difficult example group loss function are used to calculate the model error, and the model parameters are updated according to the error through the back propagation algorithm to optimize the performance of the basic feature extraction network, the cyclic semantic aggregation module, and the difficult example optimization module. Among them, the modality alignment loss function calculates and optimizes the probability distribution difference between the two modalities, uses KL divergence to gradually reduce the modality difference, and realizes effective alignment between modalities through weights that grow linearly in time steps; the group identity loss function strengthens the model's learning of identity information in the difficult sample set by calculating the cross entropy between the confidence score and the identity ID, so as to improve the model's recognition accuracy of individual identities; the group center separation loss function promotes the model to learn the distinguishing features between samples by increasing the distance between the features of the difficult sample in the embedding space, so as to enhance the recognition ability of difficult examples; the difficult example group loss function, in the difficult example retraining strategy, further improves the model's ability to distinguish difficult examples by screening the difficult example samples of each ID and performing additional training. The training result meets the preset convergence condition, which can be that the error value calculated by the loss function is less than a preset threshold, or the number of iterations reaches a predetermined value.

[0058] In a preferred embodiment, it also includes implementing a switcher strategy to alternately apply a perspective balanced sampler (VB) and a confidence-based hard example retraining strategy (CHSR) during the training process; wherein, in the conventional training stage, the model receives a multi-perspective balanced mini-batch data set from the perspective balanced sampler (VB) as input, and performs basic feature extraction and semantic feature optimization; then, the model calculates the classification confidence score, and uses the hard example retraining strategy (CHSR) to filter the hard example sample set according to the confidence vector; then, the switcher switches the model input to the hard example sample set, and performs targeted retraining with the hard example sample set to enhance the model's ability to recognize hard examples.

[0059] An embodiment of the present invention further provides a cross-modal pedestrian re-identification method, which implements cross-modal pedestrian re-identification using a model trained by the cross-modal pedestrian re-identification training method of any of the aforementioned embodiments.

[0060] The method of the embodiment of the present invention effectively overcomes the limitations of traditional methods, especially in solving semantic mismatch, insufficient optimization of hard examples, and perspective imbalance. The embodiment significantly enhances the performance and robustness of the model in the cross-modal pedestrian re-identification task through a carefully designed semantic alignment and hard example optimization framework (SAHSR). In order to verify the effectiveness of the proposed method, the present invention conducted extensive experiments on benchmark datasets in the VI-ReID field such as SYSU-MM01 and RegDB (further described later). The experimental results show that the present method significantly surpasses the existing technology in both Rank-1 accuracy and mean average precision (mAP) and achieves the highest performance. The present invention has a wide range of applicability in practical applications and is suitable for a variety of multimodal pedestrian re-identification scenarios, especially in key areas such as intelligent monitoring and public safety, and can provide strong technical support for building efficient and reliable monitoring systems.

[0061] The specific embodiments and experimental verification of the present invention are further described below.

[0062] An efficient training method for cross-modal person re-identification is proposed. The semantic alignment and hard example optimization framework (SAHSR) is designed to significantly improve the performance and robustness of the person re-identification model. The algorithm specifically includes: view-balanced sampler (VB), basic feature extraction network, recurrent semantic aggregation module (RSA) and confidence-based hard example retraining strategy (CHSR). The following is a detailed description of the implementation principle and process of each step with the help of the attached figures.

[0063] refer to Figure 1 The pedestrian re-identification model training method designed by the present invention includes the following steps S1-S5:

[0064] Step S1. First, preprocess the pedestrian image data in visible light and infrared modes, including normalization, resolution adjustment, and data enhancement operations (such as random cropping and flipping). Then, use the designed perspective balanced sampler (VB) to sample the preprocessed data to construct a small batch data set for subsequent training. Traditional sampling methods are prone to bias problems in training samples due to uneven distribution of camera perspectives. For this reason, the present invention proposes an improved perspective balanced sampler. The specific operation of the VB sampler is: first randomly select pedestrian samples of P identity categories, and then for each identity, uniformly sample K samples from images of different camera perspectives to construct a multi-perspective balanced small batch data. This design effectively reduces the performance fluctuations caused by uneven perspectives, makes full use of samples from each camera perspective, and improves the generalization ability of the model.

[0065] Step S2: Input the small batch data sampled by VB into the basic feature extraction network. The network adopts the classic ResNet-50 structure, which contains multiple convolution modules. Each module consists of a convolution layer, a batch normalization layer, and a ReLU activation function, and effectively alleviates the gradient vanishing problem through residual connections. After the input image is processed by the network, a semantically rich high-dimensional feature map is generated, which provides a feature basis for subsequent modules.

[0066] Step S3, further process the feature map using the recurrent semantic aggregation module (RSA) to generate a semantic feature representation that combines global and local features. Specifically, the RSA module first divides the feature map into several local blocks and extracts local features through average pooling; then, these local features are sequentially input into the bidirectional long short-term memory network (BiLSTM), and local semantic associations are gradually fused during the sequence processing. At the same time, global features are extracted through global average pooling. Finally, the local fused features are concatenated with the global features to generate an optimized feature vector. In order to enhance the fusion effect of local features, the present invention designs a modal alignment loss function (L ma ), by minimizing the KL divergence between the visible light and infrared modal feature distributions, the modal differences are significantly reduced during feature aggregation, and the consistency of cross-modal features is improved. First, the Euclidean distance matrix of the two modalities is calculated and the cross-modal probability distribution corresponding to the distance matrix is ​​obtained through the softmax function. and The modality alignment loss function (L ma ):

[0067]

[0068] Among them, KL refers to KL divergence, g(t) is the weight that grows linearly with the time step, which is used to gradually align the two modes when RSA aggregates local features to reduce the modal difference.

[0069] Step S4, further optimize the model's ability to distinguish difficult samples through the confidence-based difficult example retraining strategy (CHSR). The specific operation is: using the feature vector generated in step S3, the classification confidence of each input image is calculated by the classifier. The CHSR strategy selects the negative samples (i.e., difficult examples) with the highest confidence for each ID from the global data set, resamples to construct a difficult sample set, and performs additional training on it to enhance the recognition ability of the model. In the new round of retraining, the CHSR strategy will sample the corresponding difficult sample set for each ID in the previous round of training. In this process, in order to further enhance the model's ability to distinguish difficult samples, the present invention proposes a group identity loss function and a group center separation loss function. The group identity loss function calculates the cross entropy function of the confidence score and the identity ID on the difficult sample set, so that the model can further learn the identity information in the retraining stage; the group center separation loss function calculates the center separation loss function proposed in the existing research on the difficult sample set, increases the distance between the sample point feature and the corresponding difficult sample feature in the embedding space, so as to achieve the difference between the samples. In the actual training process, we designed a switcher strategy to apply the CHSR strategy. During regular training, the model input comes from the sampling results of the VB sampler. After a regular training iteration, the model will calculate the confidence vector, and the CHSR sampler will sample a set of hard sample sets based on the confidence vector. After that, the switcher will switch the model input to the hard sample set and use the input for retraining. The switcher will switch the model input alternately between the VB sampler and the CHSR sampler.

[0070] Step S5: input the training data generated in steps S1 to S4 into the model, and optimize the model parameters through iteration until the training converges. The convergence judgment criteria include: the model loss function value is lower than the set threshold or the number of training iterations reaches the set maximum value. After the training is completed, the obtained model can be directly applied to the pedestrian re-identification task in visible light and infrared modalities.

[0071] refer to Figure 2 In the embodiment shown, the basic feature extraction network adopts the ResNet-50 structure ( Figure 3 ). The network includes an initial convolution module, 4 residual modules and a global average pooling layer. After the image is input, it is processed by the initial convolution module, the residual module and the global average pooling layer in turn, and finally the classification result is output through the fully connected layer. The initial convolution module consists of a 7×7 convolution layer and a pooling layer. The residual module contains multiple bottleneck residual blocks, each of which includes 1×1, 3×3 and 1×1 convolution layers, and the direct addition of input and output is realized through residual connection, giving full play to the role of residual learning and effectively improving training efficiency and model performance.

[0072] In order to verify the effectiveness of the semantic alignment and hard example optimization method for cross-modal person re-identification proposed in this paper, comparative experiments were conducted with the current advanced methods and baseline methods in the field. The experimental results are summarized in Table 1 and Table 2. Table 1 shows the performance comparison of several existing advanced methods, and Table 2 shows the performance difference with the baseline method.

[0073] The baseline method uses the unmodified ResNet-50 network, and the experimental data sets select the SYSU-MM01 and RegDB data sets that are commonly used in this field. The evaluation indicators use mean average precision (mAP) and Rank-1 accuracy (R1). mAP measures the overall performance of the retrieval system, which represents the average precision of all relevant results of the query image target in the retrieval list, and comprehensively evaluates the retrieval effect of the model from different ranking positions. Rank-1 accuracy (R1) evaluates the correct matching probability of the model ranking first among all candidate targets, reflecting the accuracy of the top-level results.

[0074] Table 1

[0075]

[0076]

[0077] Table 2

[0078]

[0079] From the comparison results in Table 1, it can be seen that the model proposed in the present invention significantly exceeds the existing most advanced methods in terms of mAP and R1 indicators, achieving the highest accuracy in this field, which fully proves the effectiveness and advancement of the proposed method. At the same time, the comparative experimental results in Table 2 further verify the contribution of each module of the present invention: the perspective balanced sampler (VB), the recurrent semantic aggregation module (RSA) and the confidence-based difficult example retraining strategy (CHSR) all bring significant performance improvements, and the combined application of the three further enhances the model performance.

[0080] The above verification results show that the present invention has significant advantages in improving the overall performance and robustness of the cross-modal pedestrian re-identification task.

[0081] In summary, by introducing the innovative semantic alignment and hard example optimization framework (SAHSR), the embodiments of the present invention provide an efficient and robust training method for cross-modal pedestrian re-identification, significantly improve the performance of the model, and have important application value in the fields of intelligent monitoring and public safety.

[0082] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0083] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0084] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0085] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0086] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0087] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0088] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0089] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0090] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0091] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0092] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0093] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0094] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A training method for cross-modal person re-identification, characterized in that: The steps include: S1. Perform data preprocessing on pedestrian images in visible light and infrared modalities. Use the view balanced sampler (VB) to construct a multi-view balanced mini-batch dataset from each modality by sampling the camera view distribution evenly, so as to balance the data distribution of different camera views. S2, input small batches of data into the basic feature extraction network to generate semantically rich high-dimensional feature maps; S3. Optimize the high-dimensional feature map using the recurrent semantic aggregation module (RSA), where the high-dimensional feature map is divided into local blocks, and the local blocks are gradually aggregated through the bidirectional long short-term memory network (BiLSTM) to generate a global and local semantic feature representation, and optimize the consistency of cross-modal feature distribution by constructing a modality alignment loss function; The training data generated in this way is continuously input into the model, the overall loss function is calculated, and the model parameters are iteratively optimized until the training meets the preset convergence conditions.

2. The cross-modal person re-identification training method according to claim 1, characterized in that: In step S2, the basic feature extraction network adopts the ResNet-50 model.

3. The cross-modal person re-identification training method according to claim 1 or 2, characterized in that: In step S3, the recurrent semantic aggregation module (RSA) optimizes the high-dimensional feature graph by the following steps: S31, dividing the high-dimensional feature map into several local blocks, and extracting local features by average pooling; S32, local blocks are aggregated to capture semantic associations through a bidirectional long short-term memory network (BiLSTM), and BiLSTM gradually fuses local semantic associations during sequence processing; S33, extracting global features from the entire feature map through global average pooling; S34, concatenating local features and global features to form a complete semantic feature representation; S35. For the complete semantic feature representation obtained by concatenation, the consistency of cross-modal features is optimized through the modal alignment loss function.

4. The cross-modal person re-identification training method according to any one of claims 1 to 3, characterized in that: The following steps are also included: S4. Calculate the classification confidence score of the sample, filter out the hard negative samples according to the confidence, retrain the filtered hard samples using the confidence-based hard retraining strategy (CHSR), and optimize the training process of the hard samples based on the group identity loss and the group center separation loss.

5. The cross-modal person re-identification training method according to claim 4, characterized in that: Step S4 specifically includes: S41, calculating the classification confidence score of the training sample through the classifier, and selecting the negative sample with the highest confidence from the global data set as the difficult example for retraining; S42. Use group identity loss and group center separation loss to optimize the relationship between difficult negative samples and original samples.

6. The cross-modal person re-identification training method according to claim 4 or 5, characterized in that: During the training process, the model error is calculated using the group identity loss function, the group center separation loss function, the modality alignment loss function, and the hard example group loss function, and the model parameters are updated according to the error through the back propagation algorithm to optimize the performance of the basic feature extraction network, the cyclic semantic aggregation module, and the hard example optimization module; Among them, the modal alignment loss function calculates and optimizes the probability distribution difference between the two modalities, uses the KL divergence to gradually reduce the modal difference, and realizes effective alignment between the modalities through the weight that grows linearly in the time step; the group identity loss function strengthens the model's learning of identity information in the difficult sample set by calculating the cross entropy between the confidence score and the identity ID, so as to improve the model's recognition accuracy of individual identities; the group center separation loss function promotes the model to learn the discriminative features between samples by increasing the distance between the features of difficult sample samples in the embedding space, so as to enhance the recognition ability of difficult examples; the difficult example group loss function, in the difficult example retraining strategy, further improves the model's ability to discriminate difficult examples by screening difficult example samples for each ID and performing additional training.

7. The cross-modal person re-identification training method according to any one of claims 1 to 6, characterized in that: It also includes implementing a switcher strategy to alternately apply a view-balanced sampler (VB) and a confidence-based hard example retraining strategy (CHSR) during training; wherein, in the conventional training stage, the model receives a multi-view balanced mini-batch data set from the view-balanced sampler (VB) as input, and performs basic feature extraction and semantic feature optimization; then, the model calculates the classification confidence score, and uses the hard example retraining strategy (CHSR) to filter the hard example sample set according to the confidence vector; then, the switcher switches the model input to the hard example sample set, and performs targeted retraining with the hard example sample set to enhance the model's ability to recognize hard examples.

8. A cross-modal person re-identification method, characterized in that: Cross-modal pedestrian re-identification is achieved by using a model trained using the cross-modal pedestrian re-identification training method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images