A privacy-protected pedestrian re-identification method, system and device

Abstract

The present application relates to a kind of privacy protection pedestrian re-identification method, comprising the following steps: S1: data preparation: S2: pre-processing stage;S3: frequency domain conversion stage: the feature set obtained in S2 is converted to X h,w DCT coefficient in frequency domain after conversion is represented, and converted frequency domain picture is divided into key channel and non-key channel;S4: feature enhancement stage: S5: the cross-entropy overall loss in the pedestrian re-identification task of calculation model and the overall loss of generator;S6: supplement to the model by the anonymized low-frequency information sent by client, after supplementing information, the model is again carried out pedestrian re-identification task, and the cross-entropy loss and generator overall loss are calculated;S7: repeat S4-S6 until the generated model test result meets the expectation or reaches the training number.

Description

Technical Field

[0001] The present invention relates to the field of pedestrian recognition, and in particular to a privacy-protected pedestrian re-identification method, system, and device. Background Art

[0002] As person re-identification tasks are increasingly applied to real-world scenarios (such as airports and passenger stations), large amounts of pedestrian data are being used to train recognition models. Pedestrian data provides rich feature information for training models, helping to improve the accuracy and robustness of recognition algorithms. By analyzing pedestrian data, models can identify abnormal behavior patterns, prevent crime, and enhance public safety. However, sensitive data containing pedestrian privacy faces the risk of being leaked, potentially leading to the unauthorized misuse or disclosure of sensitive personal information. To address this issue, anonymous person re-identification, as a new solution, has garnered widespread attention in recent years. This approach encrypts pedestrian data, removing or obfuscating personally identifiable information, such as facial features, to reduce the risk of privacy leaks and ensure security during storage and transmission. While this approach initially protects real-world data, the obfuscation operation can result in the loss of significant feature information in pedestrian datasets, potentially rendering them unusable and reducing the model's recognition capabilities. Privacy protection is not the only goal in anonymous person re-identification; the ultimate goal is to maintain the model's recognition capabilities even after privacy protection. Frequency domain processing, one of the most advanced anonymization technologies, demonstrates significant potential for data privacy protection. This technology converts raw pedestrian images into frequency domain images, extracting non-identifiable features such as gait and body shape for re-identification. Combined with privacy-preserving algorithms like differential privacy, it adds noise to the frequency domain images, mitigating the risk of privacy breaches while maintaining data usability.

[0003] Despite years of research and development in the field of anonymous person re-identification, current frequency-domain processing-based research primarily focuses on extracting high-frequency information. While this model reduces the risk of privacy leaks by extracting only high-frequency information, which is difficult for the human eye to discern, the model loses focus on effective visual features at key locations after discarding low-frequency information, which can easily lead to a decrease in recognition accuracy. For example, facial, clothing, and body shape information together constitute complete pedestrian data. However, once the model captures a person's outline, it cannot be directly applied to recognition tasks because it cannot fully capture the specific features of these key locations, and high-frequency noise interferes with the model, limiting its ability to re-identify pedestrians.

[0004] Existing anonymous person re-identification methods based on frequency-domain processing underutilize low-frequency information. Much of the low-frequency information, which contains valuable visual features, is removed and not properly reused. These methods compromise model performance to a certain extent, preventing the model from fully utilizing all available information to optimize recognition accuracy. They often prioritize protecting private data while ignoring the potential impact on recognition accuracy. Existing methods primarily focus on the privacy protection benefits of high-frequency information but fail to optimize for potential high-frequency noise, which can interfere with the model's recognition capabilities. Furthermore, because high-frequency information lacks more specific features, the re-identification performance of using only high-frequency information for recognition is significantly lower than that of the original image. Existing methods fail to apply rigorous privacy-preserving techniques to person re-identification tasks. Common anonymous person re-identification techniques include key strategies. Due to their reversibility, once the key rules are cracked, they present a high risk of privacy leakage from malicious attacks such as generation attacks. Summary of the Invention

[0005] The anonymous pedestrian re-identification method based on frequency domain processing in the present invention not only protects pedestrian privacy data, but also maintains a high recognition level of the model. It not only uses frequency domain processing, but also applies privacy protection algorithms. Excessive privacy protection usually causes a certain performance degradation. Therefore, how to balance data privacy and model utility becomes an issue that cannot be ignored. Different from the existing methods, we introduce a new method that combines high-frequency information and low-frequency information in a federated learning framework to reduce noise interference. While improving the accuracy of model recognition, it provides stricter data privacy protection for the pedestrian re-identification model. The specific technical solution is as follows:

[0006] A privacy-preserving person re-identification method comprises the following steps:

[0007] S1: Data preparation: Sample T frames of images from the pedestrian video sequence as video clips for model input

[0008] S2: Preprocessing stage: Using the discreteness of DCT, the convolutional neural network backbone network is used to extract the spatial features of each frame image obtained in S1 to obtain the feature map. The obtained feature set is expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively;

[0009] S3: Frequency domain conversion stage: X h,wRepresents the DCT coefficient in the frequency domain after conversion, m(i, j) represents the coefficient of the feature map in the spatial domain, representing the row and column in the frequency domain respectively; that is, the value of the feature map m at position (i, j) corresponds to the frequency response value (h, w) of the DCT spectrum X; and the converted frequency domain image is divided into key channels and non-key channels

[0010] S4: Feature enhancement stage: The attention score is constructed by calculating the L2 norm of the channel dimension, and then the attention map is generated through normalization to highlight the most important areas or features in each frame of the video;

[0011] S5: Calculate the overall cross entropy loss and the overall loss of the generator in the pedestrian re-identification task;

[0012] S6: Generator stage: The model will be supplemented with the anonymized low-frequency information sent by the client. After the supplementary information, the model will once again perform the pedestrian re-identification task and calculate its cross entropy loss and the overall generator loss

[0013] S7: Repeat S4-S6 until the model test results meet expectations or the training times are reached.

[0014] Optionally, the feature set in step S2 is calculated using the following formula:

[0015]

[0016] Among them, X h,w represents the DCT coefficient in the frequency domain after conversion, m(i, j) represents the coefficient of the feature map in the spatial domain, and represents the row and column in the frequency domain respectively. That is, the value of feature map m at position (i, j) corresponds to the frequency response value (h, w) of DCT spectrum X.

[0017] Optionally, step S3 specifically includes the following steps:

[0018] S31: Calculate the channel energy of each frequency band at the maximum response value X:

[0019]

[0020] where represents the c-th channel capability of the K-th frequency band, Represents the DCT coefficient of the cth channel

[0021] S32: Determine the threshold value of threshold, according to C key ={c|E c >threshold} rule, the converted frequency domain image is divided into key channels and non-key channels, where the key channels contain low-frequency information and the non-key channels contain high-frequency information. keyRepresents the set of channels selected as key channels.

[0022] S33: High-frequency information is transmitted to the server, and low-frequency information is transmitted to the client for segmented image representation for model training and inference: Server: Client:

[0023] Optionally, the threshold is determined by first selecting the five channels with the highest energy from each of the Y, Cb, and Cr components based on the energy calculated for each frequency band as key channels, for a total of 15 channels. The remaining channels are considered non-key channels. Next, the precision-recall curve for the re-identification task is calculated and analyzed after the classified information is sent to the server. Similarly, the precision and recall for different threshold groupings are calculated, and finally, a threshold is selected that maximizes precision without reducing recall.

[0024] Optionally, step S4 specifically includes the following steps:

[0025] S41: Calculate the attention score of the high-frequency information frequency band entering the server

[0026] S42: Aggregate features from different frames and frequency bands to extract discriminative information from each frame:

[0027] Optionally, step S4 further includes:

[0028] During the training phase, the lowest frequency component in the frequency domain is used to enhance the representation of shared features in the video sequence, focusing on extracting common features across the entire video sequence to enhance the model's ability to identify pedestrians. By introducing frame-level attention mapping, the model is able to learn the importance of each frame in the video and use it to enhance the representation of shared features within the sequence:

[0029] A f =σ(Wight·m f +b)

[0030] Among them, A f represents the attention map of the f-th frame, σ represents the sigmoid activation function, Wight and b represent the learned weights and biases respectively, and m f represents the feature map of the f-th frame.

[0031] Feature map of the entire sequence:

[0032]

[0033] where m ′ denotes the feature map of the whole video sequence, T denotes the number of frames in the video sequence, and denotes element-wise multiplication. In the training stage, the difference of inter-frame attention maps is used as a regularization term to reduce the common information of frames in the video, and the inter-frame regularization term is:

[0034]

[0035] where R(m ′ ) denotes the regularization term, which is used to reduce the difference between adjacent inter-frame attention maps, and ‖·‖ F denotes the Frobenius norm.

[0036]

[0037] where m shared denotes the shared feature of the whole video sequence, denotes the lowest frequency component of the f-th frame.

[0038] Optionally, the step S6 specifically comprises the following steps:

[0039] S61: Extract the low-frequency information feature map of the entering client, average the channel, and generate a feature mask: M mask = σ(W mask · F client + b mask )

[0040] S62: Transmit the feature mask as supplementary information into the server end to realize feature transfer: F s ′ erver = F server + M norm ⊙ F server

[0041] S63: The video image after the supplementary information is subjected to GAN-based anonymization to generate an updated image, and the generated data set continues to be trained and tested for the pedestrian re-identification task.

[0042] Optionally, the method for generating the anonymization updated image is:

[0043] The anonymization updated image is generated by learning a mapping function G: X→Y, where X is the updated pedestrian image set, and Y is the image set after anonymization processing. This mapping is learned through adversarial training, so that the generated anonymization image can further protect the privacy of pedestrians, while retaining enough information to facilitate the pedestrian re-identification (Re-ID) task;

[0044] The anonymization process involves a generator G X and a discriminator D Y Generator G X Responsible for converting the original image into anonymized image, the discriminator D Y Try to distinguish between real images and generated anonymized images.

[0045] A privacy-preserving person re-identification system, comprising:

[0046] The data preparation module is used to sample T frames of images from the pedestrian video sequence as video clips for model input

[0047] The preprocessing module is used to use the discreteness of DCT and use the convolutional neural network backbone network to extract the spatial features of each frame image obtained by S1 to obtain the feature map. The obtained feature set is expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively;

[0048] Frequency domain conversion module; used to h,w Represent the DCT coefficients in the frequency domain after frequency domain conversion, and divide the converted frequency domain image into key channels and non-key channels;

[0049] Feature Enhancement Module: This module is used to construct an attention score by calculating the L2 norm of the channel dimension, and then generate an attention map through normalization to highlight the most important areas or features in each frame of the video; it then calculates the overall cross entropy loss in the pedestrian re-identification task and the overall loss of the generator;

[0050] Generator module: Used to supplement the anonymized low-frequency information sent by the client. After supplementing the information, the model once again performs the pedestrian re-identification task, calculates its cross-entropy loss and the overall generator loss; and repeats training until the model test results meet expectations or the training number is reached.

[0051] An electronic device, comprising a memory and a processor, wherein the memory and the processor are connected;

[0052] The memory stores computer instructions, and the processor executes the above-mentioned privacy-preserving pedestrian re-identification method by executing the computer instructions.

[0053] A computer-readable storage medium is characterized in that computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to enable a computer to execute the above-mentioned privacy-preserving pedestrian re-identification method.

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] Compared with existing methods, the anonymous pedestrian re-identification method based on frequency domain processing that we proposed realizes the protection of pedestrian data privacy in the federated learning framework without the need for additional public data and key strategies, reducing the risk of leakage after key cracking, and making full use of the iterative supplementation of low-frequency information to enrich the features of high-frequency information in the recognition task. The anonymized dataset can still be used for pedestrian re-identification tasks.

[0056] After the original image is converted into a frequency domain image, it is divided into high-frequency information channels and low-frequency information channels. The high-frequency information enters the server for recognition tasks, and the low-frequency information enters the client. The features on the client that are easily recognizable to the human eye are not directly transmitted to the server, thereby achieving privacy protection of specific features.

[0057] In order to balance data privacy protection and task applicability, the data entering the server and performing recognition tasks focuses on reducing features that are easily recognizable to the human eye. The pedestrian re-identification model learned by the attacker is difficult to recover and reconstruct because the data in the training process itself lacks many key features, thus avoiding the impact of generated attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 is a flow chart of an embodiment;

[0060] Figure 2 Schematic diagram of the model of the embodiment. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the present invention more apparent, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are illustrative and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0062] Anonymous person re-identification (Re-ID) based on frequency domain processing is a challenging privacy-preserving task for person re-ID. It aims to protect private data while maintaining a high recognition performance. Person re-identification (Re-ID) analyzes images or video data to identify and track the identities of specific individuals across multiple sensors (e.g., cameras). While this task plays a crucial role in public safety and surveillance efficiency, it also raises privacy concerns. Current Re-ID models may infringe on personal privacy when training datasets and lack transparency in the data collection process, allowing pedestrian data to be captured and analyzed without their knowledge. Because these models are designed to identify and track individuals, unauthorized access to private data such as facial and body shapes implicit in training and test datasets could potentially leak sensitive personal information. Under certain attacks, such as generation attacks, attackers could potentially steal sensitive training data from published trained Re-ID models, resulting in serious privacy issues. Furthermore, current research on frequency domain processing is primarily applied to face recognition tasks, including feature extraction, image enhancement, and noise reduction. Therefore, the present invention proposes an effective method, namely an anonymous pedestrian re-identification method based on frequency domain processing, which enables the model to maintain high recognition accuracy while protecting privacy. Specifically, we applied a federated learning framework and decomposed the original pedestrian image into high-frequency channel images and low-frequency channel images through a discrete cosine transform network (DCT). The server side processes the high-frequency channels representing abstract features, and the client side processes the low-frequency channels representing specific features. For privacy protection, after removing the specific features, the pedestrian data on the server side no longer has the specific features in human vision, effectively resisting malicious behaviors such as generation attacks. For the pedestrian re-identification task, we supplemented the server side with some low-frequency information in the client to improve the recognition accuracy.

[0063] The present invention proposes a new anonymous pedestrian re-identification model - a pedestrian re-identification framework with privacy preserving. Compared with the existing technical solutions, it has three main innovations: the first is to introduce a frequency selection module based on attention enhancement, and use channel segmentation and attention mechanism to select the representative frequency of pedestrian contours in the server that processes high-frequency information; the second is to add a content supplement module based on feature mask generation, and the high-frequency information is combined with part of the low-frequency information that has been anonymized to perform feature enhancement again, thereby improving the client's ability to judge pedestrian matching in pedestrian re-identification tasks; the third is to combine the generative adversarial network in the client that processes low-frequency information, and anonymize the low-frequency information. The anonymized data is used to supplement the server, providing privacy protection for the specific features of the original pedestrian, and can better balance the accuracy and privacy of the model in the re-identification task. The following is a detailed explanation of the methods and modules designed and used by the present invention. The scheme specifically includes the following steps, and the flow chart is as follows Figure 1 Shown, including:

[0064] S1: Data preparation: Given a video sequence of a pedestrian, assume that T frames of images are sampled as video clips for the model input. T represents the number of sampling frames

[0065] S2: In the preprocessing stage, the discreteness of DCT is used to perform spatial features on each frame of the image using the convolutional neural network backbone network, and the feature map is extracted. The feature set obtained can be expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively.

[0066] S3: In the frequency domain conversion stage, X h,w Represents the DCT coefficient in the frequency domain after conversion, m(i, j) represents the coefficient of the feature map in the spatial domain, and represents the row and column in the frequency domain respectively. That is, the value of the feature map m at position (i, j) corresponds to the frequency response value (h, w) of the DCT spectrum X. And the converted frequency domain image is divided into key channels and non-key channels

[0067] S4: In the feature enhancement stage, the L2 norm of the channel dimension is calculated to construct the attention score, and then the attention map is generated through normalization to highlight the most important areas or features in each frame of the video.

[0068] S5: Calculate the cross entropy overall loss and the overall loss of the generator in the pedestrian re-identification task

[0069] S6: In the generator stage, the model will be supplemented with the anonymized low-frequency information sent by the client. After the supplementary information, the model will once again perform the pedestrian re-identification task and calculate its cross entropy loss and the overall generator loss.

[0070] S7: Repeat S4-S6 until the model test results meet expectations or the training times are reached.

[0071] Traditional spatial domain methods may lead to the destruction of spatial relationships due to spatial alignment problems between frames, which will affect the accuracy of pedestrian recognition. Compared with the method based on spatial segmentation, FSM uses DCT-based frequency information. In the frequency domain, information exists in the form of frequency components, which do not need to be precisely aligned like pixels in the spatial domain. Therefore, even if there are slight spatial offsets or alignment problems between frames in the original video, their representation in the frequency domain can still maintain consistency, allowing the model to focus on those frequency components that are most useful for pedestrian recognition, rather than relying on spatial features that may change due to alignment problems. This effectively avoids spatial alignment problems and the destruction of spatial relationships, and improves the model's ability to understand video content. In step S2, the DCT transformation includes:

[0072] The frequency selection module uses a restricted random sampling method. Given a video sequence, it is assumed that T frames of images are sampled as video clips for the model input. T represents the number of sampled frames. Utilizing the discreteness of DCT, a convolutional neural network (CNN) backbone network is used to perform spatial feature extraction on each frame of the image, extracting the feature maps, and then converting the feature maps in the spatial domain to the frequency domain. The feature set obtained by the CNN network can be expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively. X is converted to the frequency domain using DCT, and the corresponding DCT spectrum can be expressed as a specific formula:

[0073]

[0074] Among them, X h,w represents the DCT coefficient in the frequency domain after conversion, m(i, j) represents the coefficient of the feature map in the spatial domain, and represents the row and column in the frequency domain respectively. That is, the value of feature map m at position (i, j) corresponds to the frequency response value (h, w) of DCT spectrum X.

[0075] The step S3 specifically includes the following steps:

[0076] Frequency channel splitting

[0077] Frequency channels are divided into key channels and non-key channels based on their importance. Key channels contain most of the visual information, while non-key channels are used for model training and inference on the server side to protect privacy.

[0078] In order to extract the discriminative features of video frames in the frequency domain, most methods use global average pooling or horizontal spatial segmentation as frame-level representative features, which may cause model information loss or inconsistent spatial alignment. This module retains valid information by using fine-grained frequency features instead of spatial features, increasing the valid information embedded in different frequency components, which can effectively avoid spatial alignment problems or spatial relationship destruction problems. Based on the assumption that the global average pooling result is one of the special frequency components in DCT, the formula involved is:

[0079]

[0080] in, and f(i,j) denote the DCT output and input respectively, H is the input dimension, and W is the frequency of application.

[0081] Formula (2) selects frequency equal to 0 and cosine term equal to 1. The results show that the average pooling result is equivalent to the lowest frequency component in DCT (the component with frequency 0):

[0082]

[0083] X 00 Represents the lowest frequency component in the DCT spectrum, that is, the global average pooling result.

[0084] Based on the above proof, different DCT components can be applied to explore the most discriminative information in each frame, and the lowest DCT component can be used to represent the shared information of the entire sequence. Specifically, the transformed frequency spectrum is divided into several frequency bands, and each frequency band is represented by its maximum response value. This can express effective information more compactly and help identify and extract the most discriminative features in each frame.

[0085] The entire frequency spectrum X is divided into K frequency bands, and each frequency band is represented by the maximum response value. To express:

[0086]

[0087] Among them, B k Represents the range of the kth frequency band.

[0088] Calculate the channel energy of each frequency band at the maximum response value X:

[0089]

[0090] Where, represents the c-th channel capability of the K-th frequency band. Represents the DCT coefficient of the c-th channel.

[0091] By selecting key channels, thresholds are set to determine which channels contain important visual information.

[0092] C key ={c|E c >threshold}

[0093] Among them, C key represents the set of channels selected as key channels, and threshold is a threshold used to determine which channels contain important visual information.

[0094] The threshold is determined as follows: First, based on the energy calculated for each frequency band, the five channels with the highest energy from each of the Y, Cb, and Cr components are selected as key channels, for a total of 15 channels. The remaining channels are considered non-key channels. Next, the precision-recall curve for the re-identification task is calculated and analyzed after the classified information is sent to the server. Precision and recall are calculated for different threshold groupings in this manner, and finally, a threshold is selected that maximizes precision without compromising recall.

[0095] High-frequency information transmitted to the server and low-frequency information transmitted to the client, segmented image representation for model training and inference:

[0096]

[0097] Among them, I server and I client Denotes the image representations used for model training and inference on the server and client sides, respectively. non-key represents the set of non-critical channels, C key Represents a collection of key channels.

[0098] 3) In step S4, attention enhancement uses an attention map to learn the most discriminative information in each frame. An attention score is constructed by calculating the L2 norm of the channel dimension, and then normalized to generate an attention map. This attention map highlights the most important regions or features in each frame of the video, thereby guiding the model to focus on the visual cues that are most helpful for the person re-identification task. The formula involved is as follows:

[0099] a. Calculate the attention score for each frequency band:

[0100]

[0101] Among them, S c,x,y represents the attention score at channel c and position (x,y).

[0102] b. Generate normalized attention map:

[0103]

[0104] Among them, S c ′ ,x,y represents the normalized attention score.

[0105] c. Construct a 2D attention map:

[0106] M c,x,y =S c ′ ,x,y ·X x,y

[0107] Among them, M c,x,y represents the weighted frequency response at channel c and position (x,y).

[0108] By aggregating features from different frames and frequency bands, we can extract the discriminative information of each frame while suppressing unimportant features. The extracted discriminative information can be expressed as:

[0109]

[0110] Where D represents the discriminative feature after aggregation.

[0111] During the training phase, the module uses the lowest frequency components in the frequency domain to enhance the representation of shared features in the video sequence, focusing on extracting common features across the entire video sequence to enhance the model's ability to identify pedestrians. By introducing frame-level attention mapping, the model is able to learn the importance of each frame in the video and use it to enhance the representation of shared features within the sequence:

[0112] A f =σ(Wight·m f +b)

[0113] Among them, A f represents the attention map of the f-th frame, σ represents the sigmoid activation function, Wight and b represent the learned weights and biases respectively, and m f represents the feature map of the f-th frame.

[0114] Feature map of the entire sequence:

[0115]

[0116] Among them, m ′ Represents the feature map of the entire video sequence, T represents the number of frames in the video sequence, and ⊙ represents element-by-element multiplication. During the training phase, the difference in attention maps between frames is used as a regularization term to bring the common information of frames in the video closer. The inter-frame regularization term is:

[0117]

[0118] Among them, R(m ′ ) represents the regularization term, which is used to reduce the difference in attention maps between adjacent frames, ‖·‖ F represents the Frobenius norm.

[0119]

[0120] Among them, m shared represents the shared features of the entire video sequence, Indicates the lowest frequency component of the f-th frame.

[0121] The server-side model obtains the pedestrian's outline information through the frequency selection module. However, since this information is high-frequency information that is visually unimportant, the data accessed by the server-side model may cause its attention to human features to be inaccurate, reducing its performance in pedestrian feature recognition. Therefore, an information interaction module (Interactive block) is introduced to allow attention to be transferred from the client to the server, and to compensate for the lack of visual information by passing information to the client model, thereby helping the server-side model to more accurately recognize pedestrian features. The step S6 includes the following steps:

[0122] First, the client calculates the average value of each channel from the feature map and generates a single-channel feature mask. This mask reflects the importance of different regions in the feature map. Feature mask generation:

[0123] M mask =σ(W mask ·F client +b mask )

[0124] Among them, M mask represents the feature mask, F client represents the feature map extracted by the client, W mask and b mask denote the learnable weights and biases, respectively, and σ denotes the sigmoid activation function.

[0125] The client then normalizes the feature mask to the [0,1] interval:

[0126]

[0127] where M norm represents the normalized feature mask, which is used to ensure that the value of the feature mask is within a reasonable range.

[0128] The normalized feature mask is transmitted to the server side to realize attention transfer:

[0129] F s ′ erver =F server +M norm ⊙F server

[0130] where F s ′ erver represents the updated server-side feature map, F server represents the original server-side feature map, and ⊙ represents element-wise multiplication, which is used to transfer the attention of the client to the server side.

[0131] Finally, after receiving the feature mask on the server side, it is adjusted to be consistent with the feature map size of the server-side model, then activated by a sigmoid function, and then element-level multiplication operation is performed with the server-side feature map to update the server-side feature representation.

[0132] The video image after supplementing the information is anonymized based on GAN to generate a dataset, which continues to train and test the pedestrian re-identification task. The updated representation image on the server side contains part of the high-frequency information and low-frequency information of the original pedestrian data, and there may still be sensitive information points of pedestrians. In order to avoid the leakage of privacy caused by the interception of this part of information by the attacker, the updated feature representation needs to be anonymized. The conditional generative adversarial network (conditional generative adversarial networks, cGANs) is used to realize the anonymization processing of the updated image.

[0133] The anonymized updated image generation learns a mapping function G: X→Y, where X is the updated pedestrian image set, and Y is the image set after anonymization processing. This mapping is learned through adversarial training, so that the generated anonymized image can further protect the privacy of pedestrians, while retaining enough information to facilitate the pedestrian re-identification (Re-ID) task. The anonymization process involves a generator G X and a discriminator D Y . The generator G X is responsible for converting the original image into an anonymized image, and the discriminator DY Try to distinguish between real images and generated anonymized images.

[0134] An optional embodiment of the present invention provides a computer device that performs a privacy-preserving pedestrian re-identification method. The computer device includes: one or more processors, a memory, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface).

[0135] In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple storages if desired. Similarly, multiple computer devices can be connected, with each device providing part of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system).

[0136] The processor may be a central processing unit, a network processor, or a combination thereof. The processor may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0137] The memory stores instructions executable by at least one processor, so that the at least one processor executes the pedestrian re-identification method with privacy protection shown in the above embodiment.

[0138] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory may include high-speed random access memory and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state memory device. In some optional embodiments, the memory may include a memory remotely located relative to the processor, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0139] The memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid state drive; the memory may also include a combination of the above types of memory.

[0140] The computer device also includes an input device and an output device. The processor, memory, input device and output device can be connected via a bus or other means.

[0141] The input device can receive input digital or character information and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc. The output device can include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0142] The computer device further includes a communication interface for the computer device to communicate with other devices or a communication network.

[0143] An embodiment of the present invention also provides a computer-readable storage medium, and the above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded on a storage medium, or downloaded via a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware.

[0144] The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may include a combination of the aforementioned types of memory. It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0145] The embodiments of the present application may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the above method. The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present disclosure, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language 10 or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0146] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A privacy-preserving person re-identification method, characterized in that: The steps include: S1: Data preparation: Sample T frames of images from the pedestrian video sequence as video clips for model input S2: Preprocessing stage: Using the discreteness of DCT, the convolutional neural network backbone network is used to extract the spatial features of each frame image obtained in S1 to obtain the feature map. The obtained feature set is expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively; S3: Frequency domain conversion stage: The feature set obtained in S2 is converted to the frequency domain with X h,w Represent the DCT coefficients in the frequency domain after frequency domain conversion, and divide the converted frequency domain image into key channels and non-key channels; S4: Feature enhancement stage: The attention score is constructed by calculating the L2 norm of the channel dimension, and then the attention map is generated through normalization to highlight the most important areas or features in each frame of the video; Then train the generative model; S5: Calculate the cross entropy overall loss and the overall loss of the generator in the pedestrian re-identification task of the model; S6: Supplement the model with the anonymized low-frequency information sent by the client. After supplementing the information, the model once again performs the pedestrian re-identification task and calculates its cross entropy loss and the overall loss of the generator; S7: Repeat S4-S6 until the generated model test results meet expectations or the number of training times is reached; The step S3 specifically includes the following steps: S31: Calculate the channel energy of each frequency band at the maximum response value X: where represents the c-th channel capability of the K-th frequency band, Represents the DCT coefficient of the cth channel; S32: Determine the threshold value of threshold, and then key ={c|E c >threshold} rule, the converted frequency domain image is divided into key channels and non-key channels, where the key channels contain low-frequency information and the non-key channels contain high-frequency information. key represents the set of channels selected as key channels; S33: High-frequency information is transmitted to the server, and low-frequency information is transmitted to the client for segmented image representation for model training and inference: Server: Client: The step S4 further includes: During the training phase, the lowest frequency component in the frequency domain is used to enhance the representation of shared features in the video sequence, focusing on extracting common features across the entire video sequence to enhance the model's ability to identify pedestrians. By introducing frame-level attention mapping, the model can learn the importance of each frame in the video and use it to enhance the representation of shared features within the sequence: A f =σ(Wight·m f +b) Among them, A f represents the attention map of the f-th frame, σ represents the sigmoid activation function, Wight and b represent the learned weights and biases respectively, and m f Represents the feature map of the f-th frame; Feature map of the entire sequence: Where m′ represents the feature map of the entire video sequence, T represents the number of frames in the video sequence, and ⊙ represents element-by-element multiplication. During the training phase, the difference in attention maps between frames is used as a regularization term to bring the common information of frames in the video closer. The inter-frame regularization term is: Where R(m′) represents the regularization term, which is used to reduce the difference in attention mapping between adjacent frames, ‖·‖ F represents the Frobenius norm; Among them, m shared represents the shared features of the entire video sequence, Indicates the lowest frequency component of the f-th frame.

2. The privacy-preserving person re-identification method according to claim 1, characterized in that: The feature set in step S3 is calculated using the following formula: h∈{0,1,…,H-1},w∈{0,1,…,W-1} Among them, X h,w Represents the DCT coefficient in the frequency domain after conversion, m(i, j) represents the coefficient of the feature map in the spatial domain, and represents the row and column in the frequency domain respectively; that is, the value of the feature map m at position (i, j) corresponds to the frequency response value (h, w) of the DCT spectrum X.

3. The privacy-preserving person re-identification method according to claim 1, characterized in that: The step S4 specifically includes the following steps: S41: Calculate the attention score of the high-frequency information frequency band entering the server S42: Aggregate features from different frames and frequency bands to extract discriminative information from each frame:

4. The privacy-preserving person re-identification method according to claim 1, wherein: The step S6 specifically includes the following steps: S61: Extract feature maps of low-frequency information entering the client, average the channels, and generate feature masks: M mask =σ(W mask ·F client +b mask ); S62: Transmit the feature mask as supplementary information to the server to achieve feature transfer: F′ server =F server +M norm ⊙F server ; S63: The video image after supplementing the information is anonymized and updated based on GAN, and the generated dataset is used to continue training and testing the pedestrian re-identification task.

5. The privacy-preserving person re-identification method according to claim 1, characterized in that: The method for generating the anonymized updated image is: Anonymized updated images are generated by learning a mapping function G:X→Y, where X is the updated set of pedestrian images and Y is the set of anonymized images. This mapping is learned through adversarial training, so that the generated anonymized images can further protect the privacy of pedestrians while retaining enough information to facilitate pedestrian re-identification (Re-ID) tasks. The anonymization process involves a generator G X and a discriminator D Y ; Generator G X Responsible for converting the original image into anonymized image, the discriminator D Y Try to distinguish between real images and generated anonymized images.

6. A privacy-preserving person re-identification system, characterized in that: The privacy-preserving person re-identification system implements the privacy-preserving person re-identification method according to any one of claims 1 to 5, comprising: The data preparation module is used to sample T frames of images from the pedestrian video sequence as video clips for model input The preprocessing module is used to use the discreteness of DCT and use the convolutional neural network backbone network to extract the spatial features of each frame image obtained by S1 to obtain the feature map. The obtained feature set is expressed as Where C represents the number of channels, H and W represent the height and width of the feature map respectively; Frequency domain conversion module; used to h,w Represent the DCT coefficients in the frequency domain after frequency domain conversion, and divide the converted frequency domain image into key channels and non-key channels; Feature Enhancement Module: Constructs an attention score by calculating the L2 norm of the channel dimension, and then generates an attention map through normalization to highlight the most important areas or features in each frame of the video; then trains the generative model; and then calculates the overall cross entropy loss in the pedestrian re-identification task and the overall loss of the generator; Supplementary information module: used to supplement the anonymized low-frequency information sent by the client. After the supplementary information is added, the model once again performs the pedestrian re-identification task, calculates its cross entropy loss and the overall loss of the generator; and uses the feature enhancement module and the supplementary information module to repeat training until the model test results meet expectations or the training number is reached.

7. An electronic device, characterized in that: comprising a memory and a processor, wherein the memory and the processor are connected; The memory stores computer instructions, and the processor executes the privacy-preserving pedestrian re-identification method according to any one of claims 1 to 5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the privacy-preserving pedestrian re-identification method according to any one of claims 1 to 5.

Images