Pedestrian re-identification method based on image feature fusion and coding

Abstract

The invention discloses a pedestrian re-identification method based on image feature fusion and coding, and the method comprises the steps: constructing a target detector based on a cavity convolution feature pyramid network according to a constructed mixed receptive field module and a low-layer embedded feature pyramid module; classifying and positioning the to-be-detected target images with different scales to obtain divided pedestrian images; a dense feature pyramid network DFPN constructed by a feature extraction module RVNet, a feature enhancement module I DPFM, a feature enhancement module II IRFB and a feature aggregation module Heads is adopted to extract features in a pedestrian image and enhance the features; a pedestrian feature learning method module based on multi-view comparison predictive coding is constructed and embedded into a dense feature pyramid network (DFPN), and multi-view feature reconstruction is performed on input image features by using kernel density estimation and is combined with comparison learning, so that the model learns pedestrian feature expressions of different views. The inter-class difference is increased, and the intra-class similarity is kept; and pedestrian re-identification in the image is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a pedestrian re-identification method based on image feature fusion and coding. Background Art

[0002] Different from the active cooperation when collecting human faces, cameras have great randomness when shooting human bodies. Therefore, the quality of pedestrian images collected in monitoring scenarios is very limited. On the one hand, due to the influence of camera parameters such as perspective, resolution, and shooting distance, the resolution of pedestrians in monitoring images is generally low or blurred, and the scales and postures of the same pedestrian imaged under different devices vary. On the other hand, across cameras, the appearance differences of the same pedestrian captured at different times are large (such as changing clothes); in particular, due to the wide monitoring field of view of the camera, a large amount of background noise and irrelevant information are introduced into the captured monitoring images; most importantly, due to the complex monitoring scenario, pedestrians are easily blocked by each other, or key information is covered by other objects. In a single complex scene in a fixed field-of-view monitoring environment, the actually obtained pedestrian images are often incomplete, and pedestrian images are easily blocked by other pedestrians or objects. These occlusions will affect the extraction of pedestrian semantic information by the model, cause the semantic distribution deviation of pedestrian images, and further affect the feature matching between pedestrians, resulting in a relatively serious matching deviation and poor robustness of existing methods under the occlusion dataset.

[0003] In a more complex single scene, the specific images of pedestrians cannot even be obtained, and only rely on the simple text description of pedestrian characteristics by witnesses to find the corresponding pedestrian images. The data distribution gap between text and images is huge, making the representations of the same person in different modalities very different, resulting in multimodal ambiguity. This ambiguity leads to significantly different distributions of features from different modalities, and it is difficult to align the features across modalities, affecting the cross-modal retrieval ability of the model. In an unknown complex scene in a cross-field-of-view monitoring environment, since pedestrian data belongs to private data, it is difficult to collect corresponding data in a specific scene for annotation training. Affected by factors such as illumination change, pose orientation, and background noise, there is a significant cross-domain data distribution shift between the training source domain data and the application target domain, resulting in poor generalization ability of existing re-identification models trained based on source domain data in the target domain. How to train a pedestrian re-identification model with good generalization using existing multi-source domain data is an urgent problem to be solved for pedestrian re-identification in unknown scenarios. Summary of the Invention

[0004] Objective of the Invention: To overcome the deficiencies in the prior art, the present invention provides a pedestrian re-identification method based on image feature fusion and coding. According to the object detector based on the dilated convolutional feature pyramid network, the dense feature pyramid network DFPN, and the construction of a pedestrian feature learning method module based on multi-view contrast prediction coding, the pedestrian re-identification in the image is realized, and the recognition accuracy is ensured.

[0005] Technical Solution: To achieve the above objective, a pedestrian re-identification method based on image feature fusion and coding of the present invention includes constructing an object detector based on the dilated convolutional feature pyramid network according to the constructed hybrid receptive field module and low-level embedded feature pyramid module, classifying and positioning the target images to be detected at different scales, and obtaining the divided pedestrian images; using the dense feature pyramid network DFPN constructed by the feature extraction module RVNet, the first feature enhancement module DPFM, the second feature enhancement module IRFB, and the feature aggregation module Heads to extract and enhance the features in the pedestrian images, and performing the discrimination of the pedestrian recognition task; constructing a pedestrian feature learning method module based on multi-view contrast prediction coding and embedding it into the dense feature pyramid network DFPN, using kernel density estimation to perform multi-view feature reconstruction on the input image features and combining it with contrast learning, so that the model learns the pedestrian feature expressions from different perspectives, increases the inter-class differences, and maintains the intra-class similarities.

[0006] Further, the object detector based on the dilated convolutional feature pyramid network includes a backbone network, a hybrid receptive field module HRFM, a low-level embedded feature pyramid module LEFPM, and a detection module; the backbone network performs convolutional processing on the image to be detected to obtain several feature maps at different levels, inputs the top-level feature map C5 into the hybrid receptive field module HRFM for processing, and then inputs it together with the features of other levels Figure 1 into the low-level embedded feature pyramid module LEFPM to fuse the feature maps at different levels, and obtains the fused feature map to be detected; inputs the fused feature map to be detected into the detection module, and the detection module classifies and positions the pedestrian targets to be detected at different scales in the feature map to be detected, and outputs several pedestrian images.

[0007] Further, the hybrid receptive field module HRFM includes several dilated convolutional layers with different dilation rates; inputs the top-level feature map C5 processed by the backbone network into several parallel dilated convolutional layers with different dilation rates, and splices the feature maps obtained by the dilated convolutional layers with different dilation rates, and the spliced feature map undergoes convolutional layer for feature information fusion to obtain the feature map C5' output by the hybrid receptive field module;

[0008] For the low-level embedded feature pyramid module LEFPM, first, the feature map of the current layer is fused with the high-level feature map after channel compression and upsampling, that is, element-wise and channel-wise addition is performed to form a composite feature map; the composite feature map is fused with the downsampled low-level feature map to form a hybrid feature map; after each hybrid feature map passes through a composite convolutional layer, a final feature map to be detected is generated; the composite convolutional layer is composed of a 3×3 convolutional layer, a BN layer, and a LeakyReLU activation layer in cascade.

[0009] Furthermore, the dense feature pyramid network DFPN includes a feature extraction module RVNet, a first feature enhancement module DPFM, a second feature enhancement module IRFB, and a feature aggregation module Heads; the feature extraction module RVNet extracts features from the pedestrian image to obtain several high-level feature maps; the several high-level feature maps are input into the first feature enhancement module DPFM in a densely connected manner, and the rich semantic information in the high-level feature maps is recursively passed downward, and is fused with the lower-level feature maps in a specific way to obtain a first enhanced feature map; the first enhanced feature map is input into the second feature enhancement module IRFB for processing to obtain a second enhanced feature map; the feature aggregation module Heads aggregates the second enhanced feature maps that have undergone multiple convolutions and feature enhancements to form a feature-enhanced pedestrian image for discrimination of the pedestrian recognition task.

[0010] Furthermore, the feature extraction module RVNet is composed of five Block modules of Block1 - Block5, and Block1, Block2, Block3, Block4, and Block5 output feature maps C11, feature map C12, feature map C13, feature map C14, and feature map C15 respectively; the first feature enhancement module DPFM processes the feature map C15 output by Block5 through the CCL module to obtain a feature map Feature5;

[0011] Feature5 = CCL(C15)

[0012] The deep feature map Feature5 containing rich semantics is concatenated with the feature map C14 output by Block4 in a 1:1 channel ratio through the Op1 dense connection operation to obtain a feature map Feature4;

[0013] Feature4 = [C14, Op1(Feature5)]

[0014] The feature map Feature5 and the feature map Feature4 are respectively concatenated with the feature map C13 output by Block3 through the Op2 dense connection operation and the Op3 dense connection operation to obtain a feature map Feature3;

[0015] Feature3 = [C13, Op2(Feature4), Op3(Feature5)].

[0016] Furthermore, the second feature enhancement module IRFB is improved based on the original receptive field module RFB. The 5×5 convolutional layer in the receptive field module RFB is replaced by two cascaded 3×3 convolutional layers with a stride of 1. When the feature map passes through the first 3×3 convolutional layer, the number of channels of the feature map is compressed to half of the original; when the feature map passes through the second 3×3 convolutional layer, the number of channels of the feature map is restored to the original corresponding value; and the second feature enhancement module IRFB is inserted at the intersection of Block2 and Block3 in the feature extraction module RVNet.

[0017] Furthermore, in the multi-view contrast prediction coding-based pedestrian feature learning method module, the input multi-view image data is encoded into the embedding space and multi-view coding contrast prediction is performed; the multi-view images of the same pedestrian are encoded, and the pedestrian feature encoding at any image view is obtained through kernel density estimation; through multi-view prediction, the encoding prediction results at each view are aligned with the true pedestrian features, so that the network model can adaptively align different embedding representations of the same pedestrian to maintain intra-class similarity, thereby obtaining a view-invariant pedestrian visual expression; the contrast learning method is introduced into the multi-view prediction coding process to directly encourage different pedestrian images to be separated from each other in the feature coding space, increasing the difference between the feature encodings of different pedestrian images; realizing that the intra-class features are similar and the inter-class features are distinguishable.

[0018] Furthermore, contrast learning is introduced to maximize the distance between different pedestrian images in the representation space; enabling different pedestrian images to be separated from each other in the feature space, thereby enhancing the feature similarity of the same pedestrian; the optimization objective of multi-view contrast prediction coding is expressed as:

[0019]

[0020]

[0021] In the formula, τ is a regulation parameter, K() is a kernel function, h is a window width, sim is a similarity function operation, and exp is an exponential function operation.

[0022] Beneficial effects: A pedestrian re-identification method based on image feature fusion and coding according to the present invention introduces dilated convolutional layers of different sizes into a feature pyramid to construct a hybrid receptive field module, aiming to obtain more global feature information by increasing the receptive field under the condition of controlling the number of parameters, so as to solve the problem of target occlusion; designs a low-level embedded feature pyramid module to fuse shallow feature detail information and high-level feature semantic information, improves the richness and representation ability of the feature map, and enhances the scale adaptability of the model; a dense pyramid fusion module recursively transmits rich high-level semantic information downward in a densely connected manner, improves the richness and scale diversity of the feature map, enables it to take into account both semantic information and detail information, and thus alleviates the problems of multi-scale and various postures of pedestrians; introduces an improved retina receptive field structure to effectively alleviate the problems of pedestrian blur and occlusion; multi-view coding prediction and contrast learning are used to jointly optimize the re-identification model, thereby helping the model learn a perspective-robust pedestrian feature representation. Brief Description of the Drawings

[0023] Figure 1 is the network structure diagram of an object detector based on a dilated convolutional feature pyramid network;

[0024] Figure 2 is the network structure diagram of the hybrid receptive field module HRFM;

[0025] Figure 3 is the network structure diagram of the low-level embedded feature pyramid module LEFPM;

[0026] Figure 4 is the network structure diagram based on the dense feature pyramid network DFPN;

[0027] Figure 5 is the network structure diagram of the dense pyramid fusion module DPFM;

[0028] Figure 6 is the schematic diagram of the dense connection network structure in the dense pyramid fusion module DPFM;

[0029] Figure 7 is the improved schematic diagram of the improved retina receptive field structure IRFB;

[0030] Figure 8 is the schematic diagram of the receptive fields of the RFB before improvement and the IRFB after improvement;

[0031] Figure 9 is the network structure diagram based on multi-view contrast prediction coding. Detailed Embodiment

[0032] The present invention will be further described below with reference to the accompanying drawings.

[0033] As Figure 1As shown in the figure, a pedestrian re-identification method based on image feature fusion and coding includes constructing an object detector based on an atrous convolution feature pyramid network according to the constructed hybrid receptive field module and lower-layer embedded feature pyramid module, classifying and locating target images to be detected at different scales, and obtaining segmented pedestrian images. An object detector based on an atrous convolution feature pyramid network (ACFPN); introducing atrous convolution layers of different sizes into the feature pyramid to construct a hybrid receptive field module (HRFM), aiming to obtain more global feature information by increasing the receptive field under the condition of controlling the number of parameters to solve the problem of target occlusion; secondly, improving the structure of the feature pyramid network (FPN) and designing a lower-layer embedded feature pyramid module (LEFPM) to fuse shallow feature detail information and high-layer feature semantic information, improve the richness and representational ability of the feature map, and enhance the scale adaptability of the model; aiming at the problem of missed detection that easily occurs when pedestrian targets are interfered by the background and move quickly, introducing the Anchor Free mechanism in the FCOS algorithm to effectively reduce the redundancy of candidate boxes and further improve the positioning accuracy.

[0034] As Figure 4As shown, since it is difficult for the performance of the pedestrian target detection algorithm to reach 100%, a large amount of background information will still be retained in the cropped pedestrian images, which will seriously affect the performance of the pedestrian re-identification algorithm. To alleviate the quality problem of pedestrian images and improve the accuracy of the pedestrian re-identification algorithm, a dense feature pyramid network (DFPN) constructed by a feature extraction module RVNet, a feature enhancement module I - DPFM, a feature enhancement module II - IRFB, and a feature aggregation module Heads is used to extract features from pedestrian images and enhance the features for the discrimination of pedestrian recognition tasks. The feature extraction module RVNet draws on the construction methods of residual blocks and VGGNet to design a pre - training - free residual - type backbone network (Residual VGGNet, RVNet) to extract features, which can effectively improve the migration ability of the model. Secondly, by improving the feature fusion method of the Feature Pyramid Network (FPN), a dense pyramid fusion module (DPFM) is constructed. This module recursively transmits the rich semantic information in the high - level in a dense connection manner, improving the richness and scale diversity of the feature maps, making it take into account both semantic information and detail information, thus alleviating the problems of multi - scale and various postures of pedestrians. Finally, to increase the receptive field of the feature maps, an improved retina receptive field structure (Improved RFB, IRFB) is introduced to effectively alleviate the problems of pedestrian blur and occlusion.

[0035] As Figure 9 shown, in the unknown complex scenarios of cross - view monitoring environments, since pedestrian data belongs to privacy data, it is difficult to collect corresponding data in specific scenarios for annotation training. A pedestrian feature learning method module based on multi - view contrast prediction coding is constructed and embedded in the dense feature pyramid network (DFPN) to replace the original CCL module in the dense feature pyramid network (DFPN). In view of the large difference in data styles under different cameras, kernel density estimation is used to perform multi - view feature reconstruction on the input image features and combine it with contrast learning to maximize the mutual information between different views of the same pedestrian, thereby learning view - invariant information and enabling the model to learn the pedestrian feature expressions from different perspectives. A contrast learning mechanism based on positive and negative samples is introduced in the process of multi - view coding prediction to further improve the quality of the learned image features, enhance the feature discrimination ability of the model, increase the inter - class difference and maintain the intra - class similarity. Multi - view coding prediction and contrast learning are used to jointly optimize the re - identification model, thus helping the model learn view - robust pedestrian feature expressions.

[0036] As Figure 1As shown in the figure, the object detector based on the dilated convolutional feature pyramid network is based on the one-stage fully convolutional object detection algorithm FCOS, and introduces the Anchor Free mechanism unique to FCOS. The object detector based on the dilated convolutional feature pyramid network includes a backbone network, a hybrid receptive field module HRFM, a low-level embedded feature pyramid module LEFPM, and a detection module; both the hybrid receptive field module LEFPM and the low-level embedded feature pyramid module HRFM act on the feature maps generated by the backbone network and perform different functions in the entire architecture. The backbone network performs convolutional processing on the image to be detected to obtain several feature maps at different levels. The backbone network performs convolutional processing on the image to be detected and generates feature maps with different high-level and low-level feature information, that is, several feature maps at different levels, and obtains the feature map at the C3 level, the feature map at the C4 level, and the feature map at the C5 level. The top-level feature map C5 is input into the hybrid receptive field module HRFM for processing and then input into the low-level embedded feature pyramid module LEFPM together with the feature maps of other levels to fuse the feature maps at different levels and obtain the fused feature image; the top-level feature map C5 is input into the hybrid receptive field module HRFM for processing, and the output feature map C5' of the hybrid receptive field module; the output feature map C5' of the hybrid receptive field module, the feature map at the C3 level, and the feature map at the C4 level are input into the low-level embedded feature pyramid module LEFPM to obtain the to-be-detected feature maps P5, P4, and P3 to be detected after fusion; the fused to-be-detected feature maps are input into the detection module, and by means of the combined design of sub-networks of different scales, the detection module classifies and locates the pedestrian targets to be detected at different scales in the to-be-detected feature maps and outputs several pedestrian images. Figure 1 Then, they are input into the low-level embedded feature pyramid module LEFPM to fuse the feature maps at different levels and obtain the fused feature image; the top-level feature map C5 is input into the hybrid receptive field module HRFM for processing, and the output feature map C5' of the hybrid receptive field module; the output feature map C5' of the hybrid receptive field module, the feature map at the C3 level, and the feature map at the C4 level are input into the low-level embedded feature pyramid module LEFPM to obtain the to-be-detected feature maps P5, P4, and P3 to be detected after fusion; the fused to-be-detected feature maps are input into the detection module, and by means of the combined design of sub-networks of different scales, the detection module classifies and locates the pedestrian targets to be detected at different scales in the to-be-detected feature maps and outputs several pedestrian images.

[0037] As Figure 2 shown, the hybrid receptive field module HRFM improves the network's ability to obtain global features by splicing the feature maps obtained in parallel by dilated convolutional layers with different dilation rates, and makes up for the grid effect caused by a single dilated convolution; the hybrid receptive field module HRFM uses all dilated convolutional layers, and the dilated convolution has the greatest impact on the receptive field of the top-level feature map. In order to give full play to the performance of HRFM; the standard convolution can be regarded as a special form of the dilated convolution when ar is 1, and the output of the dilated convolution is defined as:

[0038]

[0039] In the formula, H and W respectively represent the length and width of the input image or the previous layer's feature map; x(i,j) is the pixel value or feature value at the (i,j) position on the input image; ar is the dilation rate; y(i,j) is the output of the input image after passing through the dilated convolutional layer.

[0040] The hybrid receptive field module HRFM includes dilated convolutional layers with several different dilation rates, which consists of a 1×1 convolutional layer branch and three 3×3 convolutional layer branches with dilation rates of ar = 1, 2, and 4 respectively; the 1×1 convolutional layer can preserve the detailed information of the image as much as possible without changing the size of the feature map, and at the same time can control the number of channels of the feature map, reducing the subsequent computational amount; the 3×3 convolutional layer has fewer parameters, which can not only process the feature information but also further reduce the computation of the network; the 3×3 dilated convolutional layer with ar = 4 can aggregate more global context feature details, enhancing the inference ability of the model, thereby alleviating the occlusion problem of the target; at the same time, the use of multiple dilated convolutional layers further improves the adaptability of the model to targets of different scales; after the concatenated feature map, a 1×1 convolutional layer is used for feature information fusion, and the channel dimension is reduced to a specified number, improving the flexibility of the HRFM module. The top-level feature map C5 processed by the backbone network is input into dilated convolutional layers with several parallel different dilation rates, and the feature maps obtained by the dilated convolutional layers with different dilation rates are concatenated. The concatenated feature map undergoes feature information fusion through a convolutional layer, and then passes through a 1×1 convolutional layer to obtain the feature map C5′ output by the hybrid receptive field module.

[0041] As Figure 3 shown, the low-level embedded feature pyramid module LEFPM, based on FPN, further fuses the detailed information of the low level through the way of low-level embedding to achieve the double improvement of the multi-scale target detection effect and the positioning accuracy; first, the current layer feature map is fused with the high-level feature map after channel compression and upsampling, that is, element-wise and channel-wise addition to form a composite feature map; the composite feature map is fused with the downsampled low-level feature map to form a hybrid feature map; each hybrid feature map passes through a composite convolutional layer to generate the final feature map to be detected; among them, the feature map C5′ passes through a 1×1 convolutional layer to obtain the composite feature map P5′, the composite feature map P5′ is fused with the downsampled low-level feature map C4 to obtain a hybrid feature map, and the hybrid feature map passes through a composite convolutional layer to obtain the feature map P5 to be detected; the feature map C4 is fused with the upsampled high-level feature map to obtain a composite feature map, the composite feature map is fused with the downsampled low-level feature map C3 to obtain a hybrid feature map, and the hybrid feature map passes through a composite convolutional layer to obtain the feature map P4 to be detected; the feature map C3 is fused with the upsampled high-level feature map to obtain a composite feature map, the composite feature map is directly used as a hybrid feature map, and the hybrid feature map passes through a composite convolutional layer to obtain the feature map P3 to be detected. The composite convolutional layer is composed of a 3×3 convolutional layer, a BN layer, and a LeakyReLU activation layer in cascade, aiming to process the fused features, optimize the model training, and improve the non-linear expression ability of the features.

[0042] As Figure 4As shown, due to the interference of various environmental factors, the performance of the pedestrian target detection algorithm is limited. The cropped pedestrian images are still blurred, occluded, and have variable scales due to factors such as background information. The dense feature pyramid network DFPN includes a feature extraction module RVNet, a first feature enhancement module DPFM, a second feature enhancement module IRFB, and a feature aggregation module Heads; the first three are newly designed modules and perform different functions in the overall architecture. The feature extraction module RVNet is the Backbone based on the dense feature pyramid fusion network and is one of the most important modules in DFPN; RVNet consists of a total of five Block modules, namely Block1 - Block5, and each Block is composed of several designed RCB (Residual Convolution Block) units. The RCB can not only effectively control the number of channels of the feature map, but also effectively alleviate the problem of network degradation by adopting the design idea of Shortcut.

[0043] The first feature enhancement module DPFM selects multiple high-level feature maps in RVNet and, using the method of dense connection, recursively transmits the rich semantic information in the high-level feature maps downward and fuses them with the lower-level feature maps in a specific way; the fused feature maps take into account both global semantic information and local detail information, which can enhance the constraint and expression of the pedestrian saliency features.

[0044] The second feature enhancement module IRFB is improved based on RFB and inserted into a specific position in the network; different from RFB, in this project, the convolutional layer with a larger kernel size in RFB is equivalently replaced, reducing the number of parameters while adding more non-linear mappings; the improved IRFB can assist the network in effectively extracting the context feature information of the same pedestrian target and improving the discriminative ability of the model for occluded targets.

[0045] The feature aggregation module Heads consists of an adaptive average pooling layer, a batch normalization layer, and a fully connected layer; the purpose is to integrate the highly abstracted features after multiple convolutions before, and at the same time provide an interface for the distance loss Distance Loss and the classification loss Identification Loss; the pooling layer and the normalization layer can respectively play the roles of reducing the dimension of the features and obtaining a stable data distribution.

[0046] As Figure 4As shown in the figure, the input pedestrian image is processed by the CCL module and then input into the RVNet of the input value feature extraction module. The RVNet feature extraction module extracts features from the pedestrian image to obtain several high-level feature maps, namely feature map C13, feature map C14, and feature map C15. The second feature enhancement module IRFB is inserted at the intersection of Block2 and Block3 in the RVNet feature extraction module. The several high-level feature maps, namely feature map C13, feature map C14, and feature map C15, are input into the first feature enhancement module DPFM in a densely connected manner. The rich semantic information in the high-level feature maps is recursively passed downward, and is fused with the lower-level feature maps in a specific way to obtain the first enhanced feature map. The first enhanced feature map is input into the second feature enhancement module IRFB for processing to obtain the second enhanced feature map. The feature aggregation module Heads aggregates the second enhanced feature map that has undergone multiple convolutions and feature enhancements to form an enhanced pedestrian image, that is, the multi-scale dense feature map output based on the dense feature pyramid network DFPN, for the discrimination of the pedestrian recognition task.

[0047] As Figure 5 shown, dense connections are applied to DPFM to recursively pass downward the rich semantic information in the high-level feature maps. In a specific way, that is, the ratio of the number of channels of the current layer feature map to the sum of the channels of all the passed-down high-level feature maps = 1:1, it is concatenated with the lower-level feature maps, and finally only the end feature map is used for prediction. Compared with FPN, the advantages of DPFM are significant: Dense connections enable the loss function in the network to not only provide supervision signals for the final output layer, but also provide supervision signals for the non-output layers in the front of the network, that is, all the high-level feature maps recursively passed down. This is not only beneficial for the in-depth mining of high-level semantic information, but also further alleviates the degradation problem during network training. On the other hand, compared with direct addition, the Cat operation effectively preserves the integrity of the aggregated feature hierarchical relationship, taking into account both global semantic information and local detail information, thus effectively coping with appearance changes such as the scale of pedestrians under different devices.

[0048] As Figure 5 shown, the RVNet feature extraction module consists of five Block modules from Block1 to Block5, and Block1, Block2, Block3, Block4, and Block5 output feature map C11, feature map C12, feature map C13, feature map C14, and feature map C15 respectively. The first feature enhancement module DPFM processes the feature map C15 output by Block5 through the CCL module to obtain the feature map Feature5;

[0049] Feature5 = CCL(C15)

[0050] The deep feature map Feature5 with rich semantics is concatenated with the feature map C14 output by Block4 through the dense connection operation of Op1 in a 1:1 channel ratio. The concatenation uses the Cat operation to obtain the feature map Feature4;

[0051] Feature4 = [C14, Op1(Feature5)]

[0052] In the formula, [] represents the Cat concatenation operation;

[0053] The feature map Feature5 and the feature map Feature4 are respectively concatenated with the feature map C13 output by Block3 through the dense connection operations of Op2 and Op3 in a 1:1 channel ratio. The concatenation uses the Cat operation to obtain the feature map Feature3, which is the first enhanced feature map;

[0054] Feature3 = [C13, Op2(Feature4), Op3(Feature5)].

[0055] As Figure 6 shown, the dense connection operation of Op1 concatenates and fuses the feature map Feature5 with the feature map C14 after passing through the CCL module and 2×Upsampling upsampling. The dense connection operation of Op2 concatenates and fuses the feature map Feature5 with the feature map C13 after passing through the CCL module and 2×Upsampling upsampling. The dense connection operation of Op1 concatenates and fuses the feature map Feature4 with the feature map C13 after passing through the CCL module and 4×Upsampling upsampling.

[0056] As Figure 7 shown, the second feature enhancement module IRFB is improved based on the original receptive field module RFB. The 5×5 convolutional layer in the receptive field module RFB is replaced by two consecutive 3×3 convolutional layers with a stride of 1. When the feature map passes through the first 3×3 convolutional layer, the number of channels of the feature map is compressed to half of the original; when the feature map passes through the second 3×3 convolutional layer, the number of channels of the feature map is restored to the original corresponding value; and the second feature enhancement module IRFB is inserted at the intersection of Block2 and Block3 in the feature extraction module RVNet, which can enhance the sensitivity of the IRFB module to the scale change of the feature map.

[0057] As Figure 8 shown, the improved IRFB has the following two advantages. Thanks to the mapping of two activation functions, the improved IRFB has more non-linearity; while keeping the receptive field size unchanged, the improved IRFB has fewer parameters. The calculation formula of the receptive field R is:

[0058]

[0059] Wherein, R k-1 is the receptive field size of the (k - 1)-th layer; f k is the convolutional kernel size of the k-th layer; s i represents the stride of the i-th layer;

[0060] The receptive field sizes of the RFB before and after improvement are calculated according to the above formula as follows:

[0061] R RFB = 5, R IRFB = 3 + (3 - 1)×1 = 5;

[0062] The number of parameters S of the two is calculated as:

[0063] S RFB = 5×5×c×2c = 50c 2

[0064]

[0065] It can be known through calculation and comparison that the number of parameters of the improved IRFB becomes about 1 / 4 of the original RFB, greatly reducing the number of parameters and effectively improving the calculation efficiency.

[0066] The feature aggregation module Heads aggregates the second enhanced feature map that has undergone multiple convolutions and feature enhancements to form a feature-enhanced pedestrian image, which is the multi-scale dense feature map output based on the dense feature pyramid network DFPN; at the same time, it provides interfaces for the distance loss Distance Loss and the classification loss Identification Loss; the end feature map that combines shallow detail features and deep semantic features is output for discrimination of the pedestrian re-identification task.

[0067] The CCL (Cross-view Contrastive Learning) module is a method for pedestrian feature learning. Its core lies in combining multi-view contrastive learning and predictive coding technology to enhance the model's feature representation capabilities in cross-view scenarios. It is defined as: The CCL module is a deep learning module that optimizes feature representation by comparing pedestrian features under different viewpoints and combining it with prediction tasks. The CCL module includes three functions: cross-view feature alignment, robustness enhancement, and self-supervised learning; the cross-view feature alignment is to shorten the feature distance of the same pedestrian under different viewpoints through contrastive learning, while pushing away the features of different pedestrians; the robustness enhancement is to use predictive coding tasks (such as view reconstruction or feature completion) to improve the model's adaptability to occlusion and illumination changes; the self-supervised learning can reduce dependence on labeled data and perform feature learning through the inherent correlation of multi-view data.

[0068] The CCL module typically includes core components such as a feature extractor, a multi-view contrastive learning branch, a predictive coding branch, and joint optimization. The feature extractor uses a convolutional neural network (CNN) or a Transformer as the backbone network to extract features from pedestrian images from different viewpoints (e.g., ResNet-50). The multi-view contrastive learning branch includes positive and negative sample construction and contrastive loss calculation. For positive and negative sample construction, multi-view images of the same person serve as positive sample pairs, while images of different people serve as negative sample pairs. The contrastive loss calculation uses a loss function similar to InfoNCE to maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs.

[0069] The predictive coding branch includes a perspective prediction task and a loss function; the perspective prediction task, for example, given the features of one perspective, predicts the features of another perspective through a decoder (such as using an autoencoder structure). The loss function usually uses a reconstruction loss (such as MSE) or a cross entropy loss to ensure that the predicted features are consistent with the true distribution. The joint optimization, contrast loss and prediction tasks Weighted combination, jointly optimizing model parameters, is formulated as: Among them, λ is a hyperparameter that balances the two tasks.

[0070] An improved CCL module is used to obtain a pedestrian feature learning method module based on Multi-view Predictive Coding with Contrastive Learning (MPCCL); the improved pedestrian feature learning method module based on MPCCL is inserted or embedded into the Dense Feature Pyramid Network (DFPN) to replace the original CCL module; multi-scale features are used for cross-view contrast learning and predictive coding to achieve global semantic alignment and local detail enhancement. The pedestrian feature learning method module based on MPCCL is embedded into the DFPN to perform inter-level contrast predictive coding, multi-scale feature pooling and alignment, and dynamic weight allocation. The inter-level contrast predictive coding includes cross-level contrast learning and predictive coding tasks; in cross-level contrast learning, cross-level contrast is performed on feature maps of different scales (such as C13 and C15) to force the model to learn the consistency between views; in the predictive coding task, the distribution of high-level features is predicted using low-level features (through an autoregressive model or Transformer), or the features of the missing view are reconstructed. The multi-scale feature pooling and alignment includes global pooling (such as GMP / GAP) of the multi-scale features output by the DFPN to generate feature vectors of a unified dimension; cross-view contrast learning is performed on the pooled features, for example, different views of the same pedestrian are used as positive sample pairs; additional prediction tasks, for example, predicting the features of another view based on the features of the current view through a decoder network. The dynamic weight allocation dynamically adjusts the weights of the contrast loss and the prediction task according to the importance of the feature levels (such as high-level features being more sensitive to semantics), and is formulated as: where αi and βi are weight coefficients of different levels.

[0071] such as Figure 9As shown, the pedestrian feature learning method module based on Multi-view Predictive Coding with Contrastive Learning (MPCCL) encodes the input multi-view image data into the embedding space and performs multi-view encoding contrast prediction. The specific multi-view encoding contrast prediction process is as follows: encode the multi-view images of the same pedestrian, and obtain the pedestrian feature encoding of the pedestrian at any image view through Kernel Density Estimation (KDE). In order to make full use of multi-view information, through multi-view prediction, align the encoding prediction results of each view with the real pedestrian features, so that the network model can adaptively align different embedding representations of the same pedestrian to maintain intra-class similarity, thereby obtaining a view-invariant pedestrian visual expression. Introduce the contrastive learning method into the multi-view prediction encoding process to further improve the quality of representation learning. Directly encourage different pedestrian images to be separated from each other in the feature encoding space, and increase the difference between the feature encodings of different pedestrian images. Multi-view prediction coding and contrastive learning are used to optimize the deep network model simultaneously, so as to achieve the similarity of intra-class features and the distinctiveness of inter-class features. By introducing the information interaction of multiple views, it is possible to effectively learn the pedestrian visual expressions under different views and reduce the influence brought by camera and pedestrian appearance changes.

[0072] The kernel density estimation randomly samples B samples from different camera views i to form a set of pedestrian image sets, denoted as n is the number of cameras with different views; then, the non-linear feature extractor g enc Performs feature encoding on the input observation sequence (b = 1, 2, …, and i = 1, 2, …, n) to obtain the encoded feature sequence b represents the number of images of different views of the same camera. Regard the encoded features of the same pedestrian as random variables, and assume that these encoded features are independently and identically distributed.

[0073] Assume is an independently and identically distributed sample sampled from the data distribution with the density function f(); the density function f() at any sample point can be calculated through kernel density estimation:

[0074]

[0075] In the formula, K() is the kernel function, h is the window width. Commonly used kernel functions include the Epanechnikov kernel density function, Gaussian kernel function, etc., and n is the number of cameras with different views.

[0076] By using the above kernel density estimation, non-parametric regression y can be performed:

[0077]

[0078] where y i is the response variable; since in this method the covariates can be regarded as the response variable, thus we get:

[0079]

[0080] For any given feature vector use the above formula to obtain its encoded prediction result To ensure the consistency of information between the image encoding prediction result and the true image encoding, it is achieved by minimizing the following loss function:

[0081]

[0082]

[0083] where ||·|| is the norm of the vector, such as the Euclidean norm (the square root of the sum of the squares of the elements of the vector), which is used to measure the length of the vector or the distance between two vectors; represents the estimated value and the true value the distance between them, and is measured by the norm; the final →0 means that under certain conditions (such as n→∞, h→0 and satisfying the appropriate asymptotic relationship), the limit of this expression is 0, which means that as the sample size increases and the window width changes reasonably, the distance between the estimated value and the true value tends to 0, that is, the estimation is consistent; through multi-view prediction, the encoded prediction results under each view are aligned with the true pedestrian features, so that the network model can adaptively align different embedding representations of the same pedestrian to maintain intra-class similarity, thereby obtaining a pedestrian visual expression invariant to the view.

[0084] Each feature encoding contains the recognition information from the input image, and the similarity between these features is calculated by the Euclidean distance; intuitively, the method based on kernel density estimation KDE is to linearly combine a set of multi-view image encodings with their corresponding probability weights to predict the target image encoding where the probability weight represents the contribution degree of the multi-view encoding to the prediction of the target encoding; in this way, the proposed method can adaptively reconstruct the target encoding to learn a pedestrian feature expression invariant to the view.

[0085] The multi-view encoding prediction process aims to learn an embedded representation space where different image representations of the same pedestrian have the same semantics. To make the model discriminative for different pedestrian images, contrast learning is introduced, i.e., a contrast learning mechanism based on positive and negative samples; contrast learning is used to maximize the distance between different pedestrian images in the representation space; different pedestrian images are separated from each other in the feature space, thereby enhancing the feature similarity of the same pedestrian; when calculating the distance between different feature vectors, Euclidean distance and cosine distance are two commonly used metrics. For the convenience of implementing contrast learning, cosine distance is used to measure the distance between different feature vectors, and the optimization objective can be expressed in the following form:

[0086]

[0087] In the formula, τ is the adjustment parameter, K() is the kernel function, h is the window width, and Vi is the set of image features of camera i; sim is the operation of the similarity function, which is used to measure the similarity between two feature vectors; exp is the abbreviation of the exponential function operation, representing the exponential operation with the natural constant e (approximately equal to 2.71828) as the base, is the negative sample, and log is the logarithmic operation.

[0088] The target encoded feature from Vj is used as the anchor point, and its positive samples i (i≠j) are enumerated from the encoded feature set V and negative samples When the anchor point traverses the entire set Vi, i = 1, 2,..., n, the optimization objective of multi-view contrast prediction encoding is expressed as:

[0089]

[0090] In the formula, Vi is the set of image features of camera i, Vj is the set of target encoded features, B is the total number of images of different perspectives of the same camera, and b is the b-th image of different perspectives of the same camera;

[0091] The above formula performs encoding contrast prediction encoding on the images of each pedestrian from multiple perspectives, fully utilizes the information of different camera perspectives in the pedestrian data for feature prediction and contrast; directly encourages different pedestrian images to be separated from each other in the feature encoding space, increasing the difference between the feature encodings of different pedestrian images; can learn the dependency relationships between different features and obtain pedestrian features with strong robustness to perspective changes.

[0092] Person re-identification includes three steps: feature extraction, distance measurement, and sorting and matching. The specific implementation process is as follows: Using representation learning methods, visual features of pedestrians are obtained, usually features learned through manually designed features or deep learning methods for pedestrian representation. Based on the extraction of pedestrian visual features, a distance measurement strategy is usually adopted to measure the similarity between features, so as to measure the differences between different pedestrian images, and then narrow the distance between features of pedestrians with the same identity, while expanding the differences between features of pedestrians with different identities. In the final stage, the features of the pedestrian images to be queried are compared with the features of the pedestrian images existing in the retrieval library, and sorted by distance, so as to form a sorted list of pedestrian pictures arranged from high to low based on feature similarity. On the basis of this sorted list, the sorted list of pedestrian images can also be optimized through re-ranking or relevance feedback methods to obtain a better sorted list of pedestrian images. The key points of person re-identification technology lie in how to extract robust and discriminative pedestrian visual features, and how to calculate and optimize the similarity measurement between these features.

[0093] Embodiment

[0094] In the specific person re-identification task of the present invention, the implementation steps are as follows:

[0095] Step1. Process the input image data through an object detector based on the Atrous Convolution Feature Pyramid Network (ACFPN); classify and locate the target images to be detected at different scales to obtain the divided pedestrian images;

[0096] Step2. Input the multi-view images of the same pedestrian (such as the front, side, and back views taken by different cameras) or the multi-scale images of the same pedestrian (different resolutions or different cropped areas) into the Dense Feature Pyramid Network (DFPN);

[0097] Step3. Process the input image based on the Dense Feature Pyramid Network (DFPN), extract features from the input image data based on the Dense Feature Pyramid Network (DFPN), and generate multi-scale dense feature maps {F1, F2,..., Fn};

[0098] Step4. Operate the multi-view contrast prediction coding module,

[0099] (1) Contrast learning branch, calculate the contrast loss (such as InfoNCE) for the features of different views of the same pedestrian (such as Fi of view A and Fj of view B); and perform negative sample sampling on the features of different pedestrians;

[0100] (2) Prediction coding branch, based on the feature Fi of view A, predict the feature of view B through the decoder; calculate the reconstruction loss between the predicted feature and the real feature (such as MSE or cosine similarity);

[0101] Step 5. Joint optimization. The total loss function is the weighted sum of the contrastive loss and the prediction loss. Backpropagation is used to update the parameters of the pedestrian feature learning method module based on the dense feature pyramid network (DFPN) and the multi-view contrastive prediction coding (MPCCL).

[0102] Step 6. Finally, based on the dense feature pyramid network (DFPN), a multi-scale dense feature map is output, which is the pedestrian image with enhanced features, thereby realizing the pedestrian re-identification task.

[0103] The above is only a description of the preferred embodiments of the present invention. Those of ordinary skill in the art can make several modifications and optimizations based on the above disclosure without departing from the basic principle content. These improvements and optimizations should be regarded as the protection scope understood by the present invention.

Claims

1. A pedestrian re-identification method based on image feature fusion and coding, characterized in that: An object detector based on a dilated convolutional feature pyramid network is constructed, including a constructed hybrid receptive field module and a low-level embedded feature pyramid module, to classify and locate target images to be detected at different scales, and obtain segmented pedestrian images. A dense feature pyramid network DFPN constructed by a feature extraction module RVNet, a feature enhancement module I DPFM, a feature enhancement module II IRFB, and a feature aggregation module Heads is used to extract and enhance features in the pedestrian images for the discrimination of pedestrian recognition tasks. A pedestrian feature learning method module based on multi-view contrast prediction coding is constructed and embedded in the dense feature pyramid network DFPN. Kernel density estimation is used to perform multi-view feature reconstruction on the input image features and combine it with contrast learning, enabling the model to learn pedestrian feature expressions from different perspectives, increasing the inter-class differences, and maintaining intra-class similarities.

2. The pedestrian re-identification method based on image feature fusion and coding according to claim 1, characterized in that: The object detector based on the dilated convolutional feature pyramid network includes a backbone network, a hybrid receptive field module HRFM, a low-level embedded feature pyramid module LEFPM, and a detection module. The backbone network performs convolutional processing on the image to be detected to obtain feature maps at several different levels. The top-level feature map C5 is input into the hybrid receptive field module HRFM for processing, and then together with feature maps at other levels, it is input into the low-level embedded feature pyramid module LEFPM to fuse feature maps at different levels, obtaining a fused feature map to be detected. The fused feature map to be detected is input into the detection module, and the detection module classifies and locates pedestrian targets to be detected at different scales in the feature map to be detected, and outputs several pedestrian images.

3. The pedestrian re-identification method based on image feature fusion and coding according to claim 2, wherein: The hybrid receptive field module HRFM includes several dilated convolutional layers with different dilation rates. The top-level feature map C5 processed by the backbone network is input into several parallel dilated convolutional layers with different dilation rates, and the feature maps obtained by the dilated convolutional layers with different dilation rates are concatenated. The concatenated feature map undergoes convolutional layer for feature information fusion to obtain the feature map C5′ output by the hybrid receptive field module. For the low-level embedded feature pyramid module LEFPM, first, the current layer feature map is fused with the high-level feature map after channel compression and upsampling, that is, element-wise and channel-wise addition, to form a composite feature map. The composite feature map is fused with the downsampled low-level feature map to form a hybrid feature map. Each hybrid feature map undergoes a composite convolutional layer to generate the final feature map to be detected. The composite convolutional layer is composed of a 3×3 convolutional layer, a BN layer, and a LeakyReLU activation layer in cascade.

4. A pedestrian re-identification method based on image feature fusion and coding according to claim 1, characterized in that: The described dense feature pyramid network DFPN includes a feature extraction module RVNet, a first feature enhancement module DPFM, a second feature enhancement module IRFB, and a feature aggregation module Heads; the feature extraction module RVNet extracts features from the pedestrian image to obtain several high-level feature maps; the several high-level feature maps are input into the first feature enhancement module DPFM in a densely connected manner, and the rich semantic information in the high-level feature maps is recursively passed downward, and is fused with the lower-level feature maps in a specific way to obtain a first enhanced feature map; the first enhanced feature map is input into the second feature enhancement module IRFB for processing to obtain a second enhanced feature map; the feature aggregation module Heads aggregates the second enhanced feature map that has undergone multiple convolutions and feature enhancements to form a feature-enhanced pedestrian image for discrimination in the pedestrian recognition task.

5. The pedestrian re-identification method based on image feature fusion and coding according to claim 4, wherein: The feature extraction module RVNet is composed of five Block modules, namely Block1 - Block5, and Block1, Block2, Block3, Block4, and Block5 output feature maps C11, feature map C12, feature map C13, feature map C14, and feature map C15 respectively; the first feature enhancement module DPFM processes the feature map C15 output by Block5 through the CCL module to obtain the feature map Feature5; Feature5 = CCL(C15) The deep feature map Feature5 with rich semantics is concatenated with the feature map C14 output by Block4 in a 1:1 channel ratio through the Op1 dense connection operation to obtain the feature map Feature4; Feature4 = [C14, Op1(Feature5)] The feature map Feature5 and the feature map Feature4 are respectively concatenated with the feature map C13 output by Block3 through the Op2 dense connection operation and the Op3 dense connection operation to obtain the feature map Feature3; Feature3 = [C13, Op2(Feature4), Op3(Feature5)].

6. A pedestrian re-identification method based on image feature fusion and coding according to claim 4, characterized in that: The second feature enhancement module IRFB is improved based on the original receptive field module RFB. The 5×5 convolutional layer in the receptive field module RFB is replaced by two consecutive 3×3 convolutional layers with a stride of 1. When the feature map passes through the first 3×3 convolutional layer, the number of channels of the feature map is compressed to half of the original; when the feature map passes through the second 3×3 convolutional layer, the number of channels of the feature map is restored to the original corresponding value; and the second feature enhancement module IRFB is inserted at the intersection of Block2 and Block3 in the feature extraction module RVNet.

7. A pedestrian re-identification method based on image feature fusion and coding according to claim 1, characterized in that: The module of the pedestrian feature learning method based on multi-view contrast prediction coding encodes the input multi-view image data into the embedding space and performs multi-view coding contrast prediction; encodes the multi-view images of the same pedestrian and obtains the pedestrian feature coding of the pedestrian at any image view through kernel density estimation; Align the encoded prediction results from each perspective with the true pedestrian features through multi-perspective prediction, enabling the network model to adaptively align different embedded representations of the same pedestrian to maintain intra-class similarity, thereby obtaining a perspective-invariant pedestrian visual representation; Introduce the contrastive learning method into the multi-perspective prediction encoding process, directly encouraging different pedestrian images to be separated from each other in the feature encoding space, increasing the difference between the feature encodings of different pedestrian images; achieving intra-class features with similarity and inter-class features with distinctiveness.

8. A pedestrian re-identification method based on image feature fusion and coding according to claim 7, characterized in that: Introduce contrastive learning and use it to maximize the distance between different pedestrian images in the representation space; making different pedestrian images separate from each other in the feature space, thereby enhancing the feature similarity of the same pedestrian; The optimization objective of multi-perspective contrastive prediction encoding is expressed as: In the formula, τ is the adjustment parameter, K() is the kernel function, h is the window width, sim is the similarity function operation, and exp is the exponential function operation.

Images