Cross-modal pedestrian re-identification method and system for multi-view video monitoring in coal mine

Abstract

The present application relates to the technical field of image recognition, and more particularly to a cross-modal pedestrian re-identification method and system for multi-view video monitoring in a mine, which comprises the following steps: S1, obtaining a labeled pedestrian dataset and an unlabeled pedestrian dataset to be identified; S2, constructing a ResNet50 feature extraction network and an overall network model, which comprises a connected intermediate modality generation module, a ResNet50 network embedded with a CBAM module, and a feature horizontal mapping segmentation layer with a generalized average pooling layer; the generalized average pooling layer is used for feature mapping horizontal cutting, which can retain more important information for the discrimination task; a step-by-step joint loss is used for training, so as to improve the model's recognition of the same identity and different modal center features; S3, loading the labeled pedestrian dataset to train and extract features of the constructed model, and saving the optimal model; and S4, calling the optimal model to perform pedestrian matching and query on the pedestrian dataset to be identified.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a cross-modal pedestrian re-identification method and system for multi-view video surveillance in underground mines. Background Technology

[0002] In the coal mining industry, safety management has always been a crucial issue. Due to the harsh working environment and limited lighting in coal mines, workers face significant safety risks. Cross-modal person re-identification is a method that, given a visible light or infrared image of a specific individual, attempts to find matching images belonging to the same individual in image databases of both modalities. This method, by combining high-definition vision and infrared sensing, enables real-time monitoring and identification of workers under harsh conditions, reliably identifying workers even in poorly lit underground environments or under the influence of smoke and dust. As an important supplementary technology to facial recognition for identity verification, this method can improve the accuracy of facial recognition and gait recognition technologies due to interference from the harsh underground environment of coal mines. The application of this method can improve the safety of mine workers and enhance personnel management efficiency, contributing to the further modernization and safety development of the coal mining industry.

[0003] One of the main challenges currently facing cross-modal pedestrian re-identification methods in underground mines is the modal difference between visible light and infrared images. The network may be more susceptible to the influence of certain image regions, while ignoring other regions that may contain key information. Feature extraction networks cannot effectively focus on discriminative regions and details in pedestrian images, resulting in poor recognition performance. Summary of the Invention

[0004] To overcome the technical shortcomings of existing feature extraction networks, which cannot effectively focus on discriminative regions and details in pedestrian images, resulting in poor recognition performance, this invention provides a cross-modal pedestrian re-identification method and system for multi-view video surveillance in underground mines.

[0005] This invention provides a cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines, comprising the following steps:

[0006] S1. Obtain labeled pedestrian dataset and unlabeled pedestrian dataset to be identified. The labeled pedestrian dataset includes visible light images and infrared images of the same identity ID. The pedestrian dataset to be identified includes visible light images and infrared images.

[0007] S2. Construct a ResNet50 feature extraction network with an embedded CBAM attention module and its overall network model. The overall network model includes an intermediate modality generation module, an attention-based ResNet50 network, and a feature level mapping segmentation layer that incorporates a generalized-mean pooling (GeM Pooling) layer, connected sequentially. The ResNet50 feature extraction network includes a ResNet50 backbone network for feature extraction and a CBAM attention module embedded in the ResNet50 backbone network to improve attention to key features. The CBAM attention module includes a channel attention module and a spatial attention module.

[0008] S3. Load the labeled pedestrian dataset, train and extract features from the constructed ResNet50 feature extraction network, and save the optimal model;

[0009] S31. First, feature extraction is performed using the ResNet50 feature extraction network, and the sub-steps are as follows:

[0010] S311. First, input the visible light image and infrared image of the labeled pedestrian dataset into the intermediate modality generation module. Through image encoding and decoding, generate a unified intermediate modality image. Then, represent the inputs of the visible light image and its intermediate modality image, and the infrared image and its intermediate modality image, respectively as I. VIS I VtM I IR and I ItM ;

[0011] S312, Obtain the above four inputs I VIS I VtM I ItM and I IR Then, these image data are input into a ResNet50 network with an attention mechanism for feature extraction;

[0012] S312. After the features are extracted, the generalized average pooling layer is first used to process the extracted features in the feature level mapping segmentation layer, and then the features are divided into four parts; using GeM Pooling for feature level mapping segmentation can enhance the focus on important features.

[0013] The k-th feature map after feature extraction is represented as X. k The result obtained after the generalized average pooling layer is expressed as: The calculation process of the generalized average pooling layer is as follows:

[0014]

[0015] Among them, P k It is a pooling hyperparameter that is learned during backpropagation;

[0016] S32. Using a memory mechanism, a step-by-step joint loss is introduced to train the four features obtained in S312, enhancing the accuracy of model recognition. The sub-step of introducing the step-by-step joint loss is as follows:

[0017] S21. Set a specified threshold to delay the optimization of the loss function. When the actual training epochs are less than or equal to the specified threshold, the baseline loss is applied. Train the network model, baseline loss Loss due to joint identity Cross-modal triplet loss Distribution consistency loss The baseline loss is then determined. for:

[0018]

[0019] Where λ1 and λ2 are respectively and The weight parameters; the values ​​of λ1, λ2, and λ3 are 1, 0.5, and 0.2, respectively;

[0020] S322. When the actual training epochs exceed the specified threshold, use memory to increase center-to-center loss. To align identity centers across different modalities and reduce cross-modal variance, the center of the i-th identity in the corresponding modality is:

[0021]

[0022] Where A is the number of samples in the corresponding modality, then the center of the i-th identity in the RGB modality is... The center of the i-th identity in the IR mode is Calculate the distance between the center of the i-th identity in the RGB modality and the center of the i-th identity in the IR image. Then, the average of the distances corresponding to the K identities is the center-to-center loss. Then center-to-center loss for:

[0023]

[0024] Therefore, the overall joint loss function is:

[0025]

[0026] Wherein, λ3 is Weight parameters;

[0027] S4. Call the optimal model to perform pedestrian matching query on the pedestrian dataset to be identified.

[0028] Preferably, in step S22, the ResNet50 feature extraction network with embedded CBAM attention module adopts a pre-trained model, wherein the ResNet50 backbone network retains only the first four convolutional layers and the adaptive average pooling layer, and the CBAM module is added after the first convolutional layer of the ResNet50 backbone network.

[0029] Preferably, in step S31, the joint identity loss This is the label smoothing cross-entropy loss, and its formula is:

[0030]

[0031] Where C is the number of person IDs in the labeled pedestrian dataset, y represents the person ID label, and p i This is the ID prediction logic for identity i, where ε is a small constant, set to 0.1 here;

[0032] Distribution consistency loss The formula is:

[0033] Where N is the number of I in the training batch. VIS and I ir The number of images; f(·) is the output of the proposed network;

[0034] Then the triplet loss for:

[0035]

[0036]

[0037]

[0038]

[0039] Among them, D(V) i ,I j ) is the Euclidean distance between the VIS image and the IR image, and ξ is the margin parameter, which is set to 0.3. This indicates that positive sample pairs come from the VIS and IR modes, and negative sample pairs come from the VIS mode.

[0040] This invention also provides a cross-modal pedestrian re-identification system for multi-view video surveillance in underground mines, which is built based on the cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines described in this invention, and includes:

[0041] The data acquisition module includes infrared cameras and regular cameras installed in the area to be monitored or identified;

[0042] The data preprocessing module preprocesses the input pedestrian images of different modalities;

[0043] The model building module is used to build a ResNet50 feature extraction network embedded with the CBAM attention module, and to extract feature representations of each modality image output by the data preprocessing module using the ResNet50 feature extraction network.

[0044] The network training module uses a labeled cross-modal pedestrian dataset to train the model and uses a joint loss function to establish the correspondence between images of different modalities in order to perform cross-modal pedestrian matching and re-identification and obtain the optimal model.

[0045] The testing module is used to evaluate the performance and accuracy of the optimal model obtained from the training module.

[0046] The application module is used to input the pedestrian image to be identified into the saved optimal model and match and identify the target person image information from visible light to infrared or from infrared to visible light.

[0047] Compared with existing technologies, the technical solution provided by this invention has the following advantages: This invention provides a cross-modal pedestrian re-identification method and system for multi-view video surveillance in mines. It reduces intermodal differences by generating intermediate modal images to assist the original images. A CBAM attention module is added to the feature extraction network to enhance attention to features under different modalities. Furthermore, based on the use of generalized average pooling layers, feature mapping is horizontally segmented to retain information more important for the discrimination task. Finally, by using a memory mechanism, a distributed joint training is performed using baseline loss function and center-to-center loss function. While ensuring the stability and effectiveness of model training, the model's ability to recognize the central features of different modalities of the same identity is improved, thereby significantly improving the performance of the pedestrian re-identification task. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of a cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines, as described in a certain embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the model structure constructed in step S2 of the cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines, as described in a certain embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram of the ResNet50 network model in step S2 of the cross-modal pedestrian re-identification method for multi-view video surveillance in mines, as described in a certain embodiment of the present invention.

[0053] Figure 4 This is a structural diagram of a cross-modal pedestrian re-identification system for multi-view video surveillance in underground mines, as described in a certain embodiment of the present invention. Detailed Implementation

[0054] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0055] In this description, it should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. It should also be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joint" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0056] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0057] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0058] In one embodiment, such as Figure 1 As shown, the name includes: A cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines, comprising the following steps:

[0059] S1. Obtain labeled pedestrian dataset and unlabeled pedestrian dataset to be identified. The labeled pedestrian dataset includes visible light images and infrared images of the same identity ID. The pedestrian dataset to be identified includes visible light images and infrared images.

[0060] S2. Construct a ResNet50 feature extraction network with an embedded CBAM attention module and the overall network model; the overall network model includes a sequentially connected intermediate modality generation module, an attention-based ResNet50 network, and a feature level mapping segmentation layer incorporating a generalized average pooling layer; the ResNet50 feature extraction network includes a ResNet50 backbone network for feature extraction and a CBAM attention module embedded in the ResNet50 backbone network to improve attention to key features; the CBAM attention module includes a channel attention module and a spatial attention module; embedding C... The ResNet50 feature extraction network of the BAM attention module uses a pre-trained model, where the ResNet50 backbone network retains only the first four convolutional layers and the adaptive average pooling layer. The first convolutional block in each stream is different to capture modality-specific low-level representations, while the shared middle and deep convolutional blocks learn modality-shared middle and deep representations. The CBAM attention module is added after the first convolutional layer of the ResNet50 backbone network; however, the CBAM attention module is not inserted into the block itself to ensure that the network structure within the block remains unchanged, thus guaranteeing the proper use of the pre-trained parameters.

[0061] S3. Load the labeled pedestrian dataset, train and extract features from the constructed ResNet50 feature extraction network, and save the optimal model;

[0062] S31. First, feature extraction is performed using the ResNet50 feature extraction network, and the sub-steps are as follows:

[0063] S311. First, input the visible light image and infrared image of the labeled pedestrian dataset into the intermediate modality generation module. Through image encoding and decoding, generate a unified intermediate modality image. Then, represent the inputs of the visible light image and its intermediate modality image, and the infrared image and its intermediate modality image, respectively as I. VIS I VtM I IR and I ItM ;

[0064] S312, Obtain the above four inputs I VIS I VtM I ItM and I IRThen, these image data are input into a ResNet50 network with an attention mechanism for feature extraction;

[0065] S312. After extracting the features, in the feature level mapping segmentation layer, the traditional global average pooling layer is replaced with a generalized average pooling layer. The extracted features are then processed using the generalized average pooling layer and then divided into four parts.

[0066] The k-th feature map after feature extraction is represented as X. k The result obtained after the generalized average pooling layer is expressed as: The calculation process of the generalized average pooling layer is as follows:

[0067]

[0068] Among them, P k It is a pooling hyperparameter that is learned during backpropagation;

[0069] S32. Using a memory mechanism, a step-by-step joint loss is introduced to train the four features obtained in S312. The sub-step of introducing the step-by-step joint loss is as follows:

[0070] S321. Set a specified threshold to delay the optimization of the loss function. In a specific embodiment, the specified threshold can be set to 40. When the actual training epoch is less than or equal to the specified threshold, the baseline loss is applied. Train the network model, baseline loss Loss due to joint identity Cross-modal triplet loss Distribution consistency loss The baseline loss is then determined. for:

[0071]

[0072] Where λ1 and λ2 are respectively and The weight parameters; the values ​​of λ1, λ2, and λ3 are 1, 0.5, and 0.2, respectively;

[0073] Joint identity loss This is the label smoothing cross-entropy loss, and its formula is:

[0074]

[0075] Where C is the number of person IDs in the labeled pedestrian dataset, y represents the person ID label, and p i This is the ID prediction logic for identity i, where ε is a small constant, set to 0.1 here;

[0076] Distribution consistency loss The formula is:

[0077] Where N is the number of I in the training batch. VIS and I ir The number of images; f(·) is the output of the proposed network;

[0078] Then the triplet loss for:

[0079]

[0080]

[0081]

[0082]

[0083] Among them, D(V) i ,I j ) is the Euclidean distance between the VIS image and the IR image, and ξ is the margin parameter, which is set to 0.3. This indicates that positive sample pairs come from both the VIS and IR modes, and negative sample pairs come from the VIS mode;

[0084] S322. When the actual training epochs exceed the specified threshold, use memory to increase center-to-center loss. To align identity centers across different modalities and reduce cross-modal variance, memory is used to jointly train the baseline loss and center-to-center loss, thereby improving the model's recognition accuracy. The center of the i-th identity in the corresponding modality is:

[0085]

[0086] Where A is the number of samples in the corresponding modality, then the center of the i-th identity in the RGB modality is... The center of the i-th identity in the IR mode is The loss function calculates the distance between the center of the i-th identity in the RGB modality and the center of the i-th identity in the IR image. Given the global features of the RGB and IR images, this loss function calculates the distance between the centers of the same identity in different modalities. Specifically, for each identity i, the center of that identity is calculated in both the RGB and IR images. Then, the distance between the center of identity i in the RGB image and the center of identity i in the IR image is calculated. The average of the distances for the K identities is then the center-to-center loss (C2C loss). Then center-to-center loss for:

[0087]

[0088] Therefore, the overall joint loss function is:

[0089]

[0090] Wherein, λ3 is Weight parameters;

[0091] In step S3, the deep learning framework PyTorch version 1.13.1 is used during training, and the loss function is the joint loss function mentioned above. In this specific embodiment, the model is trained for a total of 80 epochs, and a warm-up strategy is used to smooth the training gradient; the initial learning rate is set to 1×10⁻⁶. -2 Then, it linearly increases to 1×10 over 10 epochs. -1 Then, in round 20, the learning rate is decayed to 1×10. -2 It further decayed to 1×10 in the 60th round. -3 The SGD optimizer was used for optimization, and the momentum parameter was set to 0.9.

[0092] S4. Call the optimal model to perform pedestrian matching query on the pedestrian dataset to be identified; load the already trained optimal model. During the training process, this model has learned the ability to extract rich features from pedestrian images, which enables it to have good generalization ability under different sensors or different scenarios. Apply the optimal model to each image in the pedestrian dataset to be identified. Through the feature extraction function of the model, the image is converted into a high-dimensional feature vector. It is matched one by one with the feature vectors in the known pedestrian dataset to find the most similar pedestrian image.

[0093] In the method described in this invention, the center-to-center loss function in the step-by-step joint loss is introduced in step S3 to align identity centers across different modalities and reduce cross-modal variance. The center-to-center loss function minimizes the distance between each center from the same identity in different modalities. Specifically, for identity i, the center of that identity in each modality is calculated, and the distance between the centers is used to calculate the loss function. By minimizing the center distance between the same identity in each modality, cross-modal identity center alignment is achieved. Directly integrating the baseline loss function and the center-to-center loss function into the end-to-end joint learning framework is challenging. This is because the learning objectives of these two components are different, the network structure is very deep, and directly combining them can easily lead to gradient explosion. Furthermore, since the features from the two modalities of the same identity in VI-ReID are very different, the center-to-center loss function becomes unstable when crossing the two modes in the early stages of model training and fails to perform well. Features are recorded using a memory to calculate identity centers. The memory is updated as training progresses. This invention also sets a threshold to delay the optimization of the loss function until the model becomes relatively stable. The center-to-center loss function is only used to update the model when the actual training epochs exceed a specified threshold. This ensures the stability and effectiveness of model training.

[0094] Another embodiment of the present invention discloses a cross-modal pedestrian re-identification system for multi-view video surveillance in underground mines, which is built based on the cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines described in the present invention, and includes:

[0095] The data acquisition module includes infrared cameras and regular cameras installed in the area to be monitored or identified;

[0096] The data preprocessing module preprocesses the input pedestrian images of different modalities;

[0097] The model building module is used to build a ResNet50 feature extraction network embedded with the CBAM attention module, and to extract feature representations of each modality image output by the data preprocessing module using the ResNet50 feature extraction network.

[0098] The network training module uses a labeled cross-modal pedestrian dataset to train the model and uses a joint loss function to establish the correspondence between images of different modalities in order to perform cross-modal pedestrian matching and re-identification and obtain the optimal model.

[0099] The testing module is used to evaluate the performance and accuracy of the optimal model obtained from the training module.

[0100] The application module is used to input the pedestrian image to be identified into the saved optimal model and match and identify the target person image information from visible light to infrared or from infrared to visible light.

[0101] A standard camera is one that uses visible light to capture images. In a specific embodiment, a terminal device is involved, which transmits image data captured by its infrared camera and standard camera to a computer's central processing unit or data storage device via wired or wireless communication. A computer-readable storage medium is also involved, storing a computer program. When the computer processor executes the computer program, it implements the method described in this invention.

[0102] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered within the protection scope of the claims.

Claims

1. A cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines, characterized in that, Includes the following steps: S1. Obtain labeled pedestrian dataset and unlabeled pedestrian dataset to be identified. The labeled pedestrian dataset includes visible light images and infrared images of the same identity ID. The pedestrian dataset to be identified includes visible light images and infrared images. S2. Construct a ResNet50 feature extraction network with an embedded CBAM attention module and the overall network model; the overall network model includes a sequentially connected intermediate modality generation module, an attention-based ResNet50 network, and a feature level mapping segmentation layer with a generalized average pooling layer; the ResNet50 feature extraction network includes a ResNet50 backbone network for feature extraction and a CBAM attention module embedded in the ResNet50 backbone network to improve attention to key features; the CBAM attention module includes a channel attention module and a spatial attention module; S3. Load the labeled pedestrian dataset, train and extract features from the constructed ResNet50 feature extraction network, and save the optimal model; S31. First, feature extraction is performed using the ResNet50 feature extraction network, and the sub-steps are as follows: S311. First, input the visible light image and infrared image of the labeled pedestrian dataset into the intermediate modality generation module. Through image encoding and decoding, generate a unified intermediate modality image. Then, represent the inputs of the visible light image and its intermediate modality image, and the infrared image and its intermediate modality image, respectively as I. VIS I VtM I IR and I ItM ; S312, Obtain the above four inputs I VIS I VtM I ItM and I IR Then, these image data are input into a ResNet50 network with an attention mechanism for feature extraction; S312. After the features are extracted, in the feature level mapping segmentation layer, the extracted features are first processed by the generalized average pooling layer, and then the features are divided into four parts. The k-th feature map after feature extraction is represented as X. k The result obtained after the generalized average pooling layer is expressed as: The calculation process of the generalized average pooling layer is as follows: Among them, P k It is a pooling hyperparameter that is learned during backpropagation; S32. A step-by-step joint loss is introduced through a memory mechanism. The step-by-step joint loss is trained using the four features obtained in S312. The sub-step for introducing the step-by-step joint loss is as follows: S321. Set a specified threshold to delay the optimization of the loss function. When the actual training epochs are less than or equal to the specified threshold, the baseline loss is applied. Train the network model, baseline loss Loss due to joint identity Cross-modal triplet loss Distribution consistency loss The baseline loss is then determined. for: Where λ1 and λ2 are respectively and The weight parameters; the values ​​of λ1, λ2, and λ3 are 1, 0.5, and 0.2, respectively; S322. When the actual training epochs exceed the specified threshold, use memory to increase center-to-center loss. To align identity centers across different modalities and reduce cross-modal variance, the center of the i-th identity in the corresponding modality is: Where A is the number of samples in the corresponding modality, then the center of the i-th identity in the RGB modality is... The center of the i-th identity in the IR mode is Calculate the distance between the center of the i-th identity in the RGB modality and the center of the i-th identity in the IR image. Then, the average of the distances corresponding to the K identities is the center-to-center loss. Then center-to-center loss for: Therefore, the overall joint loss function is: Wherein, λ3 is Weight parameters; S4. Call the optimal model to perform pedestrian matching query on the pedestrian dataset to be identified.

2. The cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines according to claim 1, characterized in that, In step S22, the ResNet50 feature extraction network with embedded CBAM attention module adopts a pre-trained model, wherein the ResNet50 backbone network retains only the first four convolutional layers and the adaptive average pooling layer, and the CBAM module is added after the first convolutional layer of the ResNet50 backbone network.

3. The cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines according to claim 2, characterized in that, In step S31, joint identity loss This is the label smoothing cross-entropy loss, and its formula is: Where C is the number of person IDs in the labeled pedestrian dataset, y represents the person ID label, and p i This is the ID prediction logic for identity i, where ε is a small constant, set to 0.1 here; Distribution consistency loss The formula is: Where N is the number of I in the training batch. VIS and I ir The number of images; f(·) is the output of the proposed network; Then the triplet loss for: Among them, D(V) i ,I j ) is the Euclidean distance between the VIS image and the IR image, and ξ is the margin parameter, which is set to 0.

3. This indicates that positive sample pairs come from the VIS and IR modes, and negative sample pairs come from the VIS mode.

4. A cross-modal pedestrian re-identification system for multi-view video surveillance in underground mines, characterized in that, It is constructed based on the cross-modal pedestrian re-identification method for multi-view video surveillance in underground mines as described in claim 1, and includes: The data acquisition module includes infrared cameras and regular cameras installed in the area to be monitored or identified; The data preprocessing module preprocesses the input pedestrian images of different modalities; The model building module is used to build a ResNet50 feature extraction network embedded with the CBAM attention module, and to extract feature representations of each modality image output by the data preprocessing module using the ResNet50 feature extraction network. The network training module uses a labeled cross-modal pedestrian dataset to train the model and uses a joint loss function to establish the correspondence between images of different modalities in order to perform cross-modal pedestrian matching and re-identification and obtain the optimal model. The testing module is used to evaluate the performance and accuracy of the optimal model obtained from the training module. The application module is used to input the pedestrian image to be identified into the saved optimal model and match and identify the target person image information from visible light to infrared or from infrared to visible light.

Images