A personal image privacy protection method, system and electronic device

Abstract

The application discloses a personal image privacy protection method, system and electronic equipment. The method provided by the application allows a user to specify a certain person in a protected image through language expression, generates multi-scale visual features with sufficient fusion of image and text information by using a lightweight deep neural network to perform parallel coding on input reference information and personal images, generates a stable specified personal privacy protection image mask through a multi-scale feature fusion and mask positioning enhancement module in the decoding process, and in addition, introduces a balanced binary cross-entropy loss to solve the pixel imbalance problem in training, optimize the network performance, and improve the personal image privacy protection effect. The application can solve the problems of excessive protection of content and pixel imbalance in the training of the reference personal image privacy protection network in the existing personal image privacy protection technology.

Description

Technical Field

[0001] This application belongs to the field of multimodal image segmentation and image privacy protection technology, specifically relating to a method, system and electronic device for personal image privacy protection. Background Technology

[0002] The explosive growth of online platforms and social media has greatly enriched people's lives. However, while sharing personal information on social networks, a large number of personal photos are transmitted and shared online, which may lead to the leakage of personal privacy information, especially personal identification information, and this could result in potential privacy violations and infringements.

[0003] Existing image privacy protection methods are mainly divided into traditional methods and deep learning methods. Traditional privacy protection methods typically employ automatic masking strategies, such as mosaicking or blurring the entire image, which inevitably leads to significant information loss. In recent years, thanks to the development of deep learning and advancements in face detection and human pose recognition technologies, existing detection methods can automatically detect all faces and bodies in an image and then blur them to filter out private information. While these privacy protection methods can effectively protect personal privacy, they also present the problem of over-protection when users only wish to hide one or a portion of the people. One or more people in the image that are not intended to be hidden are inevitably obscured, resulting in unnecessary loss of image information. In such cases, existing methods usually require manual blurring of the image area, leading to additional manual operation costs.

[0004] Recently, with advancements in multimodal technologies, image segmentation based on referential meaning has made it possible to capture and locate corresponding regions in images based on complex linguistic expressions or audio input. This provides a potential solution for locating protected objects or privacy information in images using text or speech information. More importantly, using audio or text input offers several advantages: First, text contains rich, high-dimensional semantic information, demonstrating excellent generalization and efficient object localization capabilities in referential segmentation and semantic representation. Second, interacting with devices using audio is a part of daily life, thus providing a wide range of applications for image privacy protection based on referential meaning, such as secure video conferencing. However, existing image segmentation methods rarely consider training between referential and non-referential image pixels using binary cross-entropy (BCE) loss. Since the referred region occupies only a small proportion of the image, an imbalance in training pixels can occur. This imbalance significantly hinders the development of image privacy protection techniques, impeding network optimization and leading to inaccurate object localization, severely impacting the practical performance of the method in privacy protection applications. Summary of the Invention

[0005] This invention discloses a method, system, and electronic device for protecting personal image privacy, which can effectively solve the technical problems involved in the background art.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A method for protecting personal image privacy includes the following steps:

[0008] Step 1: Construct a privacy-preserving dataset of personal images. The dataset contains images related to people, corresponding descriptive text descriptions, and real-text masking.

[0009] Step 2: Input the image and the denotation text into the text-aware visual coding network to extract visual features enhanced by multi-scale local denotation information.

[0010] Step 3: Input the visual features enhanced with multi-scale local information into the multi-scale feature decoding network to output a high-quality image mask and obtain the specific location of the person in the image who needs protection.

[0011] Step four: Train and optimize the image based on the balanced binary cross-entropy loss function. By introducing a balance coefficient term on the basis of binary cross-entropy, the problem of pixel imbalance during network training is alleviated.

[0012] As a preferred improvement of the present invention, in step one, the dataset includes a training set, a test set, and a validation set, which are divided in a ratio of 8:1:1.

[0013] As a preferred improvement of the present invention, in step one, the dataset is divided into three categories based on the different descriptive content of the referring text, specifically including:

[0014] The first type of dataset contains short texts, less than 10 words, and includes location information words;

[0015] The second type of dataset contains texts of medium length, ranging from 10 to 15 words, and includes appearance words;

[0016] The third type of dataset provides richer textual descriptions, including location information words and appearance words.

[0017] As a preferred improvement of the present invention, in step two, the visual coding network includes an image coding module, a text coding module, and an image-text multi-scale coding module, wherein:

[0018] The image encoding module is used to extract multi-scale visual features;

[0019] The text encoding module is used to encode the input text into word vectors;

[0020] The image-text multi-scale encoding module is used to input the image modal features and text modal features extracted from the original data into a text-aware attention mechanism with a text-aware control gate, thereby enhancing the visual local information referred to in the multi-scale visual features.

[0021] As a preferred improvement of the present invention, in step three, the multi-scale feature decoding network includes a multi-scale feature fusion module and a mask localization enhancement module, wherein:

[0022] The multi-scale feature fusion module further filters, supplements, and fuses multi-scale features through a multi-scale upsampling network with a noise information filtering gate.

[0023] The masking localization enhancement module locates privacy objects based on the features extracted by the multi-scale feature fusion module.

[0024] As a preferred improvement of the present invention, the mask positioning enhancement module includes two consecutive steps:

[0025] Dual attention feature enhancement: A dual attention mechanism of channel self-attention branch and spatial self-attention branch is used to enhance local feature representation and mask localization features;

[0026] Location Mask Generation: An improved ASPP is used to capture and decode the location privacy-preserving target across multiple receptive fields at different sampling rates, generating a high-quality mask for privacy-preserving post-processing operations.

[0027] A personal image privacy protection system includes:

[0028] The data construction module is used to construct a dataset for protecting the privacy of personal images. The data in the dataset includes images related to people, corresponding descriptive text descriptions, and real-text masking.

[0029] The feature extraction module is used to input images and denotative text descriptions into a text-aware visual coding network to extract visual features enhanced with multi-scale local denotative information.

[0030] The feature decoding module is used to input visual features enhanced with multi-scale local information into the multi-scale feature decoding network, output a high-quality image mask, and obtain the specific location of the person who needs to be protected in the image.

[0031] The network training module is used to train and optimize images based on the balanced binary cross-entropy loss function. By introducing a balance coefficient term on the basis of binary cross-entropy, the problem of pixel imbalance during network training is alleviated.

[0032] An electronic device includes a processor and a memory, wherein the memory stores computer program instructions that are read by the processor and executed to perform the method.

[0033] The present invention also provides a storage medium storing instructions that, when executed by a processor, implement the method described herein.

[0034] The beneficial effects of this invention are as follows:

[0035] 1. The personal image privacy protection dataset proposed in this invention provides a new network learning paradigm for privacy protection methods based on image segmentation networks, and solves the problem of over-protection of content in existing personal image privacy protection technologies;

[0036] 2. The image-text multi-scale encoding module proposed in this invention enhances the visual local information referred to in multi-scale visual features from the perspective of text perception attention. It can better integrate multi-modal features, overcome the problem of loss of referential information, improve the detection performance of the network, and effectively solve the problem of excessive protection of non-referential content in privacy protection technology.

[0037] 3. The multi-scale feature fusion module proposed in this invention further filters, supplements and fuses multi-scale features through a designed multi-scale upsampling network with a noise information filtering gate, providing powerful visual features for locating privacy objects and reducing the probability of missed detections and false detections in the network;

[0038] 4. The mask localization enhancement module proposed in this invention focuses on using the dual attention mechanism of channel self-attention branch and spatial self-attention branch to enhance local feature representation and mask localization features, and captures and locates privacy-preserving targets in multiple scale receptive fields with different sampling rates, generating more fine-grained target object masks for privacy-preserving post-processing operations.

[0039] 5. The balanced binary cross-entropy loss proposed in this invention can alleviate the pixel imbalance problem during network training, thereby enabling the network to achieve better optimization results during training and better capture the person to be protected in the image during the prediction stage. Furthermore, this method can be easily embedded into other existing networks.

[0040] 6. The privacy-preserving network proposed in this invention adopts a pure convolutional neural network structure. Compared with the Transformer network based on high computational resource requirements, it has faster inference speed while maintaining high accuracy, and has the advantages of low computational cost and high operating efficiency. Attached Figure Description

[0041] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0042] Figure 2 A system framework diagram provided for embodiments of the present invention;

[0043] Figure 3 A structural framework diagram of an electronic device provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the test results provided in an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0046] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0047] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0048] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0050] To address the issues of over-protection of content and poor performance in existing personal image privacy protection technologies due to the imbalance between pixels in referred and non-referenced images during training, this invention proposes a personal image privacy protection method based on a loss-balanced image segmentation network. This method allows users to specify a particular person in the image for protection through textual expression, thereby resolving the problem of over-protection and saving manual operation costs. The user-inputted language and the image to be protected are encoded in parallel using a text-aware visual encoder composed of two lightweight deep neural networks. This generates multi-scale visual features that are fully integrated with the text information with minimal computational cost. During decoding, the visual features are filtered by a designed multi-scale feature fusion module, which simultaneously supplements missing features and provides a strong visual feature for the referred object. A mask localization enhancement module further transfers the decoded features into a high-quality mask. Finally, the image is fed into a post-processing program to generate a stable image for the specified personal privacy protection.

[0051] To address the pixel imbalance problem during training, a balanced binary cross-entropy loss method is proposed, which introduces two balancing coefficient terms on top of the binary cross-entropy loss. This method enables the network to focus more fairly on both referred and non-referenced regions, thereby achieving better network optimization and improving the network's effectiveness in protecting personal image privacy.

[0052] The present application will be further described below with reference to specific embodiments.

[0053] Example 1

[0054] See Figure 1 and Figure 4 As shown, the present invention provides a method for protecting personal image privacy, comprising the following steps:

[0055] Step 1: Construct a privacy-preserving dataset for the referenced personal image privacy network. This dataset includes a training set, a test set, and a validation set. The data contains images related to people, corresponding referenced text descriptions, and ground truth masking.

[0056] Specifically, the training, test, and validation sets are divided in an 8:1:1 ratio. The dataset selects human-related images, corresponding textual descriptions, and ground truth masks from RefCOCO. To expand the dataset, further human-related images are selected from MSCOCO, and textual descriptions and corresponding masks are manually annotated. The dataset contains a total of 12,014 images (416×416 pixels), 53,125 textual descriptions, and 52,323 ground truth masks.

[0057] Based on diverse practical needs, the datasets are further divided into three categories according to the different descriptive text content: Category 1, Category 2, and Category 3. Category 1 datasets contain shorter descriptive texts, fewer than 10 words, providing location information through words like "left" and "right." Category 2 datasets contain medium-length descriptive texts, ranging from 10 to 15 words, including appearance-related words such as "high" and "yellow." Category 3 datasets contain richer text descriptions, including both location information words and appearance-related words.

[0058] Step 2: Input the human-related images and denotative text descriptions into the text-aware visual coding network to extract visual features enhanced with multi-scale local denotative information.

[0059] The visual encoding network includes an image encoding module, a text encoding module, and an image-text multi-scale encoding module. The image encoding module is used to extract multi-scale visual features. The text encoding module is used to encode the input text into word vectors. The image-text multi-scale encoding module inputs the image modal features and text modal features extracted from the original data into a text perception attention feature fusion network. It enhances the visual local information referred to in the multi-scale visual features by designing a text perception attention mechanism with a text perception control gate.

[0060] Specifically, a set of image-text pairs {x n ,y n} n For the input, where the text y is... n Referring to a person in image x n Image x n Multi-scale global visual features were extracted using the image encoder DarkNet-53. Where l represents the global visual feature of the l-th layer. Text y n High-level word vector embedding features are extracted using the BiGRU text encoder. Where T is the number of words, and d is the dimension of the word embedding. For W represents the width, H represents the height, and C represents the number of channels. After obtaining the high-dimensional feature input, a text-based attention mechanism-based image-text multi-scale encoding module is further employed to enhance the visual local information of the reference. Specifically, this is achieved through a designed text-aware gate. Multiply by global visual features To generate visual features for text attention The following formula represents:

[0061]

[0062]

[0063] in, Let represent the text-aware multilayer perceptron mapping function of the l-th layer, σ represent the activation function with batch normalization, and ⊙ represent the tensor dot product.

[0064] Step 3: Input the visual features enhanced with multi-scale local information into the multi-scale feature decoding network to output a high-quality image mask in order to obtain the specific location of the person in the image who needs protection.

[0065] The multi-scale feature decoding network includes a multi-scale feature fusion module and a mask localization enhancement module. The multi-scale feature fusion module further filters, supplements, and fuses multi-scale features through a designed multi-scale upsampling network with a noise information filtering gate. The mask localization enhancement module strengthens local feature representation and mask localization features through a designed dual attention mechanism utilizing channel self-attention branch and spatial self-attention branch. It also employs an improved ASPP to capture and locate privacy-preserving targets across multiple receptive fields at different sampling rates, generating target object masks for privacy-preserving post-processing operations.

[0066] Specifically, the extracted visual features enhanced with multi-scale local information are input into the multi-scale feature fusion module and the mask localization enhancement module for decoding. The multi-scale feature fusion module, in order to provide strong visual features for locating privacy-sensitive objects, needs to better filter noise information and supplement missing features in the text visual information. Given two scale features... and First of all Perform an upsampling operation, and then... Connection generates fusion features The expression is as follows:

[0067]

[0068] This strategy may generate noisy information during the upsampling process. Therefore, based on this, the multi-scale feature fusion module further filters and supplements the multi-scale features. First, a filtering gate is designed. To filter out noise information, the expression is as follows:

[0069]

[0070]

[0071] Where, ω f It is a convolution mapping function with batch normalization. Represents the filtered features, taking into account The features may be over-filtered, so further supplementation is needed, as shown in the following formula:

[0072]

[0073] Where, ω c It is a convolutional projection, used to supplement... Missing information on The decoded output of layer l-1 is then fed into the multi-scale feature fusion module of layer l-2.

[0074] The mask localization enhancement module locates privacy objects based on the features extracted by the multi-scale feature fusion module, and includes two consecutive steps: dual attention feature enhancement and localization mask generation.

[0075] Dual-attention feature enhancement: employing two self-attention modules A s Spatial attention and Ac (channel attention) are used to enhance local feature representation, as shown below:

[0076]

[0077]

[0078] in, Is Visual features of text attention generated on the surface This is a visual characteristic of dual attention. To avoid loss of visual content, the present invention further... and Perform connections to output enhanced local visual features.

[0079] Location Mask Generation: An improved ASPP is used to capture the location of privacy-preserving targets across multiple receptive fields at different sampling rates. Furthermore, a 2x bilinear upsampling algorithm is employed to obtain a high-quality mask for privacy-preserving post-processing operations.

[0080] Step four involves training and optimizing the image based on the balanced binary cross-entropy loss function. This loss function introduces a balance coefficient term on top of the binary cross-entropy loss function to alleviate the problem of pixel imbalance during network training.

[0081] Specifically, the improved balanced binary cross-entropy loss alleviates the pixel imbalance problem in image training by theoretically exploring the loss gradient value. The classic binary cross-entropy loss measures the difference between the predicted and true probabilities of image segmentation at the pixel level, as shown in the following equation:

[0082]

[0083] in, and p n,rRepresenting the real label and image x respectively n The prediction of the r-th pixel. The goal of training the network is to predict the mask P. n Each positive pixel in the middle outputs the expected predicted value p. n,r Close to 1, similar to the true label value Consistent; on the other hand, in mask P n Output the expected prediction value p on each negative pixel outside the range. n,r =0. Therefore, the above expression can be transformed into:

[0084]

[0085] Where h(p) n,r ) and f(p n,r The terms ) represent the Taylor expansion terms for positive and negative pixels, respectively, and α represents the coefficient. During training, the network calculates a set of gradient values ​​for positive and negative pixels, and the inference method is as follows:

[0086]

[0087] Where, N p and N n These represent the number of positive and negative pixels, respectively. The formulas above reflect the direction of error propagation caused by positive and negative pixels, and are related to two terms: the quantity term N. p N n And the error term. When training approaches convergence, the value of the above formula is dominated by the quantity term. However, the segmented region usually occupies a small proportion in the image, meaning that there are far fewer positive pixels than negative pixels. Therefore, the convergence of this network is easily dominated by negative pixels, resulting in unsatisfactory performance. To solve this problem, this invention proposes a balanced binary cross-entropy loss, where h(p n,r ) and f(p n,r Two balancing terms are introduced in the paper, as follows:

[0088]

[0089]

[0090] Where ε1 and ε2 are the balancing weights for positive and negative pixels, respectively. Therefore, N p and N n They are respectively transformed into -(1+ε1)N p and (1+ε2)N p The error term remains unchanged. We set ε2∈[-1,0] to reduce the number of negative pixels and ε1∈[0,1] to increase the number of positive pixels. Therefore, the pixel imbalance problem is alleviated. The balanced binary cross-entropy expression is:

[0091]

[0092] Among them, P n For masking.

[0093] See Figure 2 As shown, the present invention also provides a personal image privacy protection system, comprising:

[0094] Data construction module 11 is used to construct a personal image privacy protection dataset, wherein the data in the dataset includes images related to people, corresponding descriptive text descriptions, and real-text masking.

[0095] The feature extraction module 12 is used to input the image and the denotative text description into the text-aware visual coding network to extract multi-scale denotative local information-enhanced visual features.

[0096] The feature decoding module 13 is used to input the visual features enhanced with multi-scale local information into the multi-scale feature decoding network, output a high-quality image mask, and obtain the specific location of the person who needs to be protected in the image.

[0097] The network training module 14 is used to train and optimize images based on the balanced binary cross-entropy loss function. By introducing a balance coefficient term on the basis of binary cross-entropy, the problem of pixel imbalance during network training is alleviated.

[0098] The present invention also provides an electronic device 21, including a memory 22 and a processor 23, wherein the memory 22 stores computer program instructions, and the computer program instructions are read by the processor 23 and executed to perform the method described thereon.

[0099] The present invention also provides a storage medium storing processor-executable instructions, which, when executed by the processor, implement the method described above. The storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, and an optical disk.

[0100] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0101] Example 2

[0102] This embodiment implements the method using an RTX 6000 GPU with 24GB of video memory. Specifically, for the encoder, the maximum sentence length T is set to 20 to generate a 1024-dimensional text vector, and three multi-scale features are generated to support decoding. For the decoder, two multi-scale feature fusion modules and one mask localization enhancement module are used to generate the mask. The encoder and decoder networks are jointly trained over 50 training epochs with a batch size of 24. The initial learning rate is set to 0.0010, decreasing to 0.0005 after 30 training epochs. Training uses the balanced binary cross-entropy loss provided by this invention, set to ε1 = 1, ε2 = -0.4, which achieves optimal performance on the validation set.

[0103] Table 1 below compares the accuracy of this invention with six advanced methods, using five metrics for performance evaluation: IoU, Prec@X, Recall, F-1, and TDIR. IoU reflects the accuracy of localization, and Prec@X represents the detection accuracy at IoU = X ∈ {0.5, 0.7, 0.9}. These metrics are widely used for reference image age segmentation. For privacy-preserving metrics, Top-1 de-identification rate (TDIR) measures the ratio of successfully de-identified faces to the total number of test images, while Recall and F-1 reflect the effectiveness of overall privacy protection. Compared to privacy-preserving methods such as InterSeg and AutoBlur, the personal image privacy-preserving method based on loss-balanced denotation image segmentation networks can automatically locate the desired target and achieve better accuracy at a limited cost.

[0104] Even among all referential segmentation methods, the method of this invention achieves the best performance across all metrics, resulting in a significant improvement in accuracy. This significant improvement stems from the proposed components, such as the multi-scale feature fusion module, the mask localization enhancement module, and the balanced binary cross-entropy loss, which enhance model training to provide robust visual features and masks.

[0105] Table 1. Accuracy comparison between the present invention and six advanced methods.

[0106] Methods IoU Prec@0.5 Prec@0.7 Prec@0.9 Recall F-1 TDIR InterSeg 58.23 56.0 42.0 6.0 78.56 77.5 66.7 AutoBlur 35.8 0.47 0.0 0.0 90.25 11.62 – CMSA 64.52 73.79 52.37 11.14 80.32 82.89 75.42 MCN 66.36 73.92 53.47 12.56 83.35 84.79 78.24 VLT 69.78 78.77 55.07 14.31 86.78 88.23 82.4 CRIS 74.25 85.52 76.48 22.43 90.34 92.98 83.5 This invention 74.72 86.18 77.17 28.34 90.67 93.52 84.3

[0107] The beneficial effects of this invention are as follows:

[0108] 1. The personal image privacy protection dataset proposed in this invention provides a new network learning paradigm for privacy protection methods based on image segmentation networks, and solves the problem of over-protection of content in existing personal image privacy protection technologies;

[0109] 2. The image-text multi-scale encoding module proposed in this invention enhances the visual local information referred to in multi-scale visual features from the perspective of text perception attention. It can better integrate multi-modal features, overcome the problem of loss of referential information, improve the detection performance of the network, and effectively solve the problem of excessive protection of non-referential content in privacy protection technology.

[0110] 3. The multi-scale feature fusion module proposed in this invention further filters, supplements and fuses multi-scale features through a designed multi-scale upsampling network with a noise information filtering gate, providing powerful visual features for locating privacy objects and reducing the probability of missed detections and false detections in the network;

[0111] 4. The mask localization enhancement module proposed in this invention focuses on using the dual attention mechanism of channel self-attention branch and spatial self-attention branch to enhance local feature representation and mask localization features, and captures and locates privacy-preserving targets in multiple scale receptive fields with different sampling rates, generating more fine-grained target object masks for privacy-preserving post-processing operations.

[0112] 5. The balanced binary cross-entropy loss proposed in this invention can alleviate the pixel imbalance problem during network training, thereby enabling the network to achieve better optimization results during training and better capture the person to be protected in the image during the prediction stage. Furthermore, this method can be easily embedded into other existing networks.

[0113] 6. The privacy-preserving network proposed in this invention adopts a pure convolutional neural network structure. Compared with the Transformer network based on high computational resource requirements, it has faster inference speed while maintaining high accuracy, and has the advantages of low computational cost and high operating efficiency.

[0114] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for protecting personal image privacy, characterized in that, Includes the following steps: Step 1: Construct a privacy-preserving dataset for referential personal images. The dataset contains images related to people, corresponding referential text descriptions, and real-text masking. Step 2: Input the image and the denotation text into the text-aware visual coding network to extract visual features enhanced by multi-scale local denotation information. Step 3: Input the visual features enhanced with multi-scale local information into the multi-scale feature decoding network to output a high-quality image mask and obtain the specific location of the person in the image who needs protection. Step 4: Train and optimize the image based on the balanced binary cross-entropy loss function. By introducing a balance coefficient term on the basis of binary cross-entropy, the problem of pixel imbalance during network training is alleviated. In step two, the visual coding network includes an image coding module, a text coding module, and an image-text multi-scale coding module, wherein: The image encoding module is used to extract multi-scale visual features; The text encoding module is used to encode the input text into word vectors; The image-text multi-scale encoding module is used to input the image modal features and text modal features extracted from the original data into a text-aware attention mechanism with a text-aware control gate, thereby enhancing the visual local information referred to in the multi-scale visual features. In step three, the multi-scale feature decoding network includes a multi-scale feature fusion module and a mask localization enhancement module, wherein: The multi-scale feature fusion module further filters, supplements, and fuses multi-scale features through a multi-scale upsampling network with a noise information filtering gate. The mask localization enhancement module locates the privacy object based on the features extracted by the multi-scale feature fusion module; The mask positioning enhancement module includes two consecutive steps: Dual attention feature enhancement: A dual attention mechanism of channel self-attention branch and spatial self-attention branch is used to enhance local feature representation and masked localization features; Location Mask Generation: An improved ASPP is used to capture and decode the location privacy-preserving target across multiple receptive fields at different sampling rates, generating a high-quality mask for privacy-preserving post-processing operations.

2. The method according to claim 1, characterized in that, In step one, the dataset includes a training set, a test set, and a validation set, which are divided in a ratio of 8:1:

1.

3. The method according to claim 2, characterized in that, In step one, the datasets are divided into three categories based on the different descriptive texts they refer to, specifically including: The first type of dataset consists of short texts, fewer than 10 words, containing location information words; The second type of dataset contains texts of medium length, ranging from 10 to 15 words, and includes appearance words; The third type of dataset provides richer textual descriptions, including location information words and appearance words.

4. A system for performing the personal image privacy protection method according to any one of claims 1-3, characterized in that, include: The data construction module is used to construct a dataset for protecting the privacy of personal images. The data in the dataset includes images related to people, corresponding descriptive text descriptions, and real-text masking. The feature extraction module is used to input images and denotative text descriptions into a text-aware visual coding network to extract visual features enhanced with multi-scale local denotative information. The feature decoding module is used to input visual features enhanced with multi-scale local information into the multi-scale feature decoding network, output a high-quality image mask, and obtain the specific location of the person who needs to be protected in the image. The network training module is used to train and optimize images based on the balanced binary cross-entropy loss function. By introducing a balance coefficient term on the basis of binary cross-entropy, the problem of pixel imbalance during network training is alleviated.

5. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores computer program instructions, which are read by the processor and executed to perform the method as described in any one of claims 1-3.

Images