A method for image privacy prediction based on scene context awareness

Abstract

The present invention belongs to image processing technology, and in particular relates to an image privacy prediction method based on scene context perception, comprising obtaining an image to be shared and the scene context information of the image, namely, the sharing time, sharing location, and target audience of the image; constructing a privacy label prediction network, and using the network to predict the privacy label of the image to be shared; constructing a cross-modal image privacy prediction network based on scene context perception, inputting the image to be shared, the privacy label of the image, and the scene context information of the image into the network to predict whether the image is a private image; the prediction model of the present invention only requires two small-scale deep neural network models to complete the privacy prediction task, which is more efficient than the existing technology, and the present invention can support personalized settings for image privacy for different users.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to an image privacy prediction method based on scene context perception. Background Art

[0002] With the increasing popularity of smartphones and other mobile devices, high-quality cameras are becoming increasingly common. Consequently, capturing images and sharing them on various social platforms has become a common part of our daily lives, with people increasingly sharing images online. Image sharing occurs not only among a group of friends but also increasingly beyond a user's social circle to engage in new social interactions. Although current social networking sites allow users to change their privacy preferences, this is often a cumbersome task for the vast majority of users, who face difficulties in assigning and managing privacy settings. When these privacy settings are inappropriate, online image sharing can lead to unwanted disclosures and privacy violations. Therefore, in our current connected world, automatically predicting the privacy of images to warn users of private or sensitive content before uploading them to social networking sites has become a necessity. Without appropriate privacy protection, shared images can reveal a great deal about a user's personal and social environment and private life, as images can intuitively tell users when and where a particular moment occurred, who was involved, and their relationships. Unfortunately, many people, especially young users of social networks, frequently share private photos of themselves, their friends, and their classmates, unaware of the potential impact of unwanted disclosures and privacy violations on their future lives.

[0003] With growing concerns about image privacy, major social networking sites have begun offering privacy settings, allowing users to manually specify their own privacy settings, such as whether images are public, private, or visible only to family and friends. Some social networking sites also offer social grouping options, allowing users to group their friends according to their social circles. Different groups have different tags, allowing users to restrict access to specific images. However, due to a lack of privacy knowledge, it is difficult for ordinary users to correctly configure privacy settings to achieve their desired level of privacy protection. Furthermore, given the large number of images shared, many users spend a considerable amount of time setting the privacy level for each image to ensure their privacy. As a result, some users are reluctant to invest time in setting their privacy preferences, or even skip the process, neglecting their privacy. This ultimately leads to privacy leaks. Although relevant privacy setting methods are gradually maturing, effective technical measures are still needed to ensure that users protect their privacy when sharing images on social media. Summary of the Invention

[0004] To address the problem of insufficient accuracy of current mainstream image privacy prediction methods, this paper proposes an image privacy prediction method based on scene context perception, which specifically includes the following steps:

[0005] Obtain the image to be shared, as well as the image's scene context information, namely, the time, location, and target audience of the image;

[0006] Build a privacy label prediction network and use it to predict the privacy labels of images to be shared;

[0007] A cross-modal image privacy prediction network based on scene context awareness is constructed. The image to be shared, the privacy label of the image, and the scene context information of the image are input into the network to predict whether the image is a private image.

[0008] Furthermore, based on historical data, obtaining the sharing time, sharing location, and target audience corresponding to the predicted tag specifically includes the following steps:

[0009] Obtain the sharing time, sharing location, and target audience of the image when it is published;

[0010] Set privacy labels, manually annotate historical data sets with single attribute labels, and calculate the privacy scores of historical data sets with single attribute labels based on sharing time, sharing location, and sharing target group;

[0011] The highest score of each single-label dataset in terms of sharing time, sharing location, and sharing target population is taken as the sensitivity corresponding to the single label.

[0012] Furthermore, if N privacy labels are set, for the nth privacy label, the single label image corresponding to the label is collected as the single label attribute set corresponding to the label, and each image in the label is manually annotated. The sharing time, sharing location and sharing target group of each image are judged and scored. The score ranges from 0 to 4 points, where 0 indicates no violation, 1 indicates slight violation, 2 indicates violation, 3 indicates relatively serious violation of privacy, and 4 indicates very serious violation of privacy.

[0013] Furthermore, sharing time includes working time and rest time, sharing locations include public places, formal places, and private places, and sharing target groups include strangers, general relationship groups, and close relationship groups.

[0014] Furthermore, when using the privacy label prediction network to predict the privacy label of the image to be shared, the privacy label prediction network performs the following prediction process:

[0015] Obtain the user's sensitivity to each tag. The more private the tag is, the higher the user's sensitivity to the tag will be.

[0016] Use the Resnet network to extract image features from images shared by users, use the softmax classifier to perform multi-label classification on the image features, and obtain the confidence score of each label;

[0017] Get the value of each label's confidence multiplied by the sensitivity corresponding to the label, and use the label corresponding to the image when the value exceeds 1 as the privacy label corresponding to the image.

[0018] Furthermore, the process of determining whether an image is a private image based on the cross-modal image privacy prediction network based on scene context awareness includes:

[0019] Based on the privacy label of the image and the scene context information of the image, obtaining an affinity matrix representing the correlation between the privacy label of the image and the scene context information of the image;

[0020] The affinity matrix is used to extract the salient information of the privacy label from the privacy label of the image, and the salient information of the scene context information is extracted from the scene context information of the image;

[0021] The private text features of the scene context information are integrated with the salient features of the privacy tag to obtain first private tag information; the private text features of the privacy tag are integrated with the salient features of the scene context information to obtain private feature information;

[0022] Fusing the first privacy label information with the image features based on cross attention to obtain the second privacy label information;

[0023] The similarity between the image feature and the second privacy tag information is used as the local similarity, the similarity between the image feature and the privacy feature information is used as the global similarity, and the weighted addition of the local similarity and the global similarity is used as the similarity between the image and the privacy information;

[0024] The image is sorted according to the similarity between the image and the private information. If the similarity exceeds the set threshold, the image is judged to be a private image, otherwise it is not a private image.

[0025] Furthermore, the process of obtaining an affinity matrix representing the correlation between the privacy label of an image and the scene context information of the image includes:

[0026] A=(R a W a )(R d W d ) T

[0027] Where A is the affinity matrix; Ra The privacy tag is encoded by word2vec and then processed by the GRU model; R d After embedding the scene context information using the sentence2vector model, a vector with the same dimension as the privacy label is obtained; W α 、W d is the mapping matrix.

[0028] Furthermore, the process of obtaining the first privacy tag information and the privacy feature information includes:

[0029]

[0030]

[0031]

[0032] in, is the privacy label feature associated with the scene context information; g r is the number of privacy label types; is the privacy text feature associated with the privacy label; R a The vector obtained after word2vec encoding of the privacy label; R d After embedding the scene context information using the sentence2vector model, a vector with the same temperature as the privacy tag is obtained; E a Indicates the first privacy label information; E d Represents privacy feature information.

[0033] Furthermore, the process of obtaining the second privacy tag information includes:

[0034] The first private label information after passing through three fully connected layers is mapped into a query vector through a trainable matrix;

[0035] Map image features into key vectors and value vectors through two trainable matrices respectively;

[0036] An attention value is calculated based on the query vector, the key vector, and the value vector, and the attention value is used as the second private label information.

[0037] Furthermore, obtaining the similarity between the image and the private information includes:

[0038] S(I,M)=w1S local (I,M)+w2S glocal (I,M)

[0039]

[0040] Sgl o cal (I,M)=cosine(Pool(E v ),Pool(E d ))

[0041] Where S(I,M) represents the similarity between the image and the private information, I represents the image feature, and M represents the private information feature; w1 and w2 are balancing factors, and w1+w2=1; sigmoid(·) represents the sigmoid activation layer; MLP(·) represents the two-layer perceptron; [|] represents the concatenation operation; Pool(·) represents the average pooling operation; E v Represents image features; E d Represents privacy feature information, Indicates the second privacy label information.

[0042] This invention establishes a cross-modal prediction network and combines the original image with scene context information to model and predict individual privacy preferences. This ensures the prediction performance of image privacy and is machine-readable, effectively solving the problem of image privacy prediction lacking personalized settings and integration with image context information. Specific beneficial effects of this invention include:

[0043] 1) The present invention has high accuracy. Specifically, the present invention can improve the accuracy of image privacy prediction by combining image features and scene context information. Furthermore, experimental verification shows that the prediction accuracy achieved by this method is higher than that of methods proposed by relevant research institutes.

[0044] 2) The present invention is highly efficient. Experiments have shown that the prediction model proposed by this method only requires two small-scale deep neural network models to complete the privacy prediction task, which is highly efficient.

[0045] 3) The present invention has strong usability and can support personalized settings of image privacy for different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of an image privacy prediction method based on scene context in the present invention;

[0047] Figure 2 Schematic diagram of cross-modal fusion based on context awareness in the present invention;

[0048] Figure 3 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] In this embodiment, the privacy prediction process mainly consists of two sub-networks, and the training process requires training of both sub-networks. The first sub-network is the image privacy label prediction network. It takes an image as input, extracts image features using the Resnt network, and performs multi-label classification using a softmax classifier. The output result is the prediction confidence of 24 privacy labels * the sensitivity of the privacy label corresponding to the image publisher;

[0051] The second sub-network is a cross-modal image privacy prediction network based on scene context awareness, such as Figure 2 The training process inputs are the image, the privacy label predicted from the image, and the privacy context information of the image, and the output is the confidence level of the privacy result of the image. The cross-attention mechanism is used to execute the information fusion module and calculate the affinity matrix between the privacy label and the privacy context. The affinity matrix represents the correlation between the privacy context and the privacy label. The calculation formula is as follows:

[0052]

[0053] Where A is the affinity matrix; R a The privacy tag is encoded by word2vec and then processed by the GRU model to obtain the feature. That is, the privacy tag is first encoded by word2vec to obtain the vector representation of the privacy tag, and the vector representation is processed by the GRU model to obtain the feature representation of the privacy tag. d After embedding the scene context information into the sentence2vector model, a vector with the same temperature as the privacy tag is obtained; W a 、W d For the mapping matrix, the purpose of the two matrices is to a 、R d Perform a linear transformation and transform R a Mapping to R d The dimension of R d Mapping to R a dimension.

[0054] To obtain the features after the privacy label and privacy context are fused, the affinity matrix A is normalized along the privacy label feature dimension to obtain the privacy label attention matrix specific to the privacy context feature. The privacy label features associated with the privacy context features are then obtained. Similarly, the privacy context features associated with the privacy label features are obtained. The corresponding features are concatenated along the last dimension to obtain the features after the privacy label and privacy context information are fused. The fused feature information is the privacy information feature. This process includes:

[0055]

[0056]

[0057]

[0058] in, is the privacy label feature associated with the scene context information; g r is the number of privacy label types; is the privacy text feature associated with the privacy label; R a The vector obtained after word2vec encoding of the privacy label; R d After embedding the scene context information using the sentence2vector model, a vector with the same temperature as the privacy tag is obtained; E a Indicates the first privacy label information; E d Represents privacy feature information.

[0059] A shared fully connected layer is used to map the image and the privacy tag features after fusion with the privacy text to the same dimension. Multi-head cross attention is used to explore the potential connection between the image and the privacy tag. The attention process is implemented as follows:

[0060]

[0061] Among them, Q(query), K(key), and V(value) are the same dimensions to which the three sets of input vector images and privacy features are mapped. The privacy label feature map is used as Q, and the image feature map is used as K and V through three different fully connected layers (the mapping process uses the learnable matrix corresponding to Q(query), K(key), and V(value) for mapping. Those skilled in the art can know how to map Q(query), K(key), and V(value) based on the principle of the attention mechanism and will not be repeated in this article). The image features related to the privacy label are obtained through this feature. The weights are generated by Q and K. After the multi-head cross-attention process, information related to the image features and the privacy label features can be extracted. Attn(Q, K, V) is the information related to the extracted features. In order to combine the image with the privacy context information and the image privacy label, the similarity between the image and the privacy information is set as local similarity and global similarity. The local similarity is the similarity between the image features and the image privacy label, and the global similarity is the similarity between the image and the image privacy context. The combination of global similarity and local similarity is the final similarity between the image and the privacy information, expressed as:

[0062] S(I,M)=w1S local (I,M)+w2S glocal (I,M)

[0063]

[0064] S gl o cal (I,M)=cosine(Pool(E v ),Pool(E d ))

[0065] Where S(I,M) represents the similarity between the image and the private information, I represents the image feature, and M represents the private information feature; w1 and w2 are balancing factors, and w1+w2=1; sigmoid(·) represents the sigmoid activation layer; MLP(·) represents the two-layer perceptron; [|] represents the concatenation operation; Pool(·) represents the average pooling operation; E v Represents image features; E d Represents privacy feature information, Indicates the second privacy label information.

[0066] The present invention proposes an image privacy prediction method based on scene context perception, which specifically includes the following steps:

[0067] Obtain the image to be shared, as well as the image's scene context information, namely, the time, location, and target audience of the image;

[0068] Build a privacy label prediction network and use it to predict the privacy labels of images to be shared;

[0069] A cross-modal image privacy prediction network based on scene context awareness is constructed. The image to be shared, the privacy label of the image, and the scene context information of the image are input into the network to predict whether the image is a private image.

[0070] In this embodiment, if Figure 1 The present invention is a context-aware image privacy prediction method, which specifically includes the following steps:

[0071] For an image to be predicted, we first collect its contextual information when it is released, perform structured modeling, and model it into data that can be used by the neural network.

[0072] The image to be predicted is input into the neural network for image feature extraction. At the same time, the scene context information of the image is fused through the cross-attention mechanism. Finally, the image and scene context information are fused using the multi-head attention mechanism to calculate the similarity, and the private content of the image is predicted based on the similarity ranking.

[0073] Collect the user's sensitivity to different scene labels. Users can set strong privacy scene labels to achieve the purpose of predicting image privacy based on their personal privacy preferences. The construction process of the deep image fusion network used in this embodiment includes:

[0074] Step 1) Constructing a prediction model for image privacy labels by connecting a feature extractor and a classifier composed of multiple convolutional dense blocks;

[0075] Step 2) The scene context information of the image is fused through the cross-attention mechanism, and finally the multi-head attention mechanism is used to fuse the image and scene context information to calculate the similarity, and the privacy content of the image is predicted based on the similarity ranking.

[0076] This embodiment adopts a method that fuses image privacy labels and image scene context information based on a cross-attention mechanism, and uses multi-head attention to remember the features of the fused image and image privacy information to achieve cross-modal fusion and calculate the similarity between the image and the privacy information. Finally, the similarities are sorted to achieve the image privacy prediction effect. The method is mainly divided into two parts. The first part uses the cross-attention mechanism to fuse the features of the image privacy label and the image privacy text. The second part uses the multi-head attention mechanism to extract the associated information between the image and the privacy information, calculate the similarity between the image and the privacy information, and sort the similarities to achieve the effect of predicting the image privacy information.

[0077] In this embodiment, two prediction networks with the same structure are selected to construct a deep image fusion network. Other networks can also be selected in the field to fuse images and scene context. The present invention does not require the specific structure of the network. In addition, any network in the prior art can be used to fuse images and scene context. The fusion can occur at the decoder or the encoder, and this invention does not impose any other limitations.

[0078] As a preferred implementation, in this embodiment, the deep image fusion network can use sub-classification networks of different types and strengths each time it is trained to obtain a corresponding deep image prediction fusion network.

[0079] The deep image prediction fusion network is constrained by the loss function, and the final prediction loss L between the privacy result and the true result is Predict Expressed as:

[0080]

[0081] Among them, L Predict is the predicted cross entropy loss function of the image privacy level, which is used to measure the accuracy of the final prediction result of the image. The true distribution of the image is y i , the network output is The total number of categories is n, which is currently 2.

[0082] This embodiment also provides a specific scheme for label setting. In this embodiment, context information is divided into three categories, namely, sharing time, sharing location, and sharing target group. Since these three types of information contain a large number of categories, sharing time is divided into two types, namely, rest time and work time, sharing locations are divided into public place, work place (office, classroom and other work and study places), rest place (home, dormitory and other rest places), and sharing target groups are divided into stranger, work partner (colleagues, superiors, classmates and other work and study partners), and Family and friends (family, friends and other people with close relationships).

[0083] We expanded the existing private image dataset to create a dataset that includes image context information. The existing dataset, VISPR2, contains 24 privacy categories: body parts, receipts, home addresses, passports, manuscripts, signatures, faces, nudity, ID cards, landmarks, usernames, names, mobile phone numbers, driver's licenses, student ID cards, prescriptions, educational background, ethnicity, bills, credit cards, fingerprints, disabilities, email addresses, and birthdays. First, we chose MTurk (Amazon Mechanical Turk) as the data collection platform. People who need certain services (called Requesters) post the tasks they need to do online, and people who want to do the tasks (called Workers) can accept the tasks and get paid. We designed an MTurk-based questionnaire as a data collection method. The most obvious privacy categories in the dataset images are divided into 24 batches, each batch has 500 images, and each questionnaire has 24 images. These images belong to different batches. The questionnaire for each image will indicate which privacy category the image belongs to to prompt the worker. The worker needs to answer three simple questions about the image, namely, under what circumstances will sharing the image violate his or her privacy, such as sharing time, sharing location, and sharing target population. At the same time, each scenario is scored, with a score of 0-4 representing the degree of privacy violation: no violation, slight violation, violation, relatively serious violation of privacy, and very serious violation of privacy. This method can establish the relationship between the 24 privacy labels and contextual information.

[0084] Secondly, the collected data set is statistically analyzed and cleaned according to the following standards: First, the scores of different context categories in each batch are statistically analyzed to obtain the total score of different context information in each batch. The highest-scoring context information combination, namely the score of the combination of sharing time, sharing location, and sharing group, is used as the privacy context information of the batch. The privacy context information is manually annotated into the image matching annotations according to the corresponding privacy category to complete the privacy context modeling of the image.

[0085] When modeling individual privacy preferences, we first collect the image publisher's sensitivity to 24 privacy categories. A score of 0-4 represents the degree of privacy violation: no violation, slight violation, violation, relatively severe violation, and very severe violation. The higher the score, the more sensitive the publisher is to the privacy attribute.

[0086] Secondly, we use the ResNet network to extract image features and train an image privacy label prediction model that can predict the confidence level of the privacy label contained in the image based on the image.

[0087] Finally, the predicted image privacy label is modeled based on the collected image publisher's sensitivity to privacy labels, that is, the confidence of the privacy label predicted by the image is * the sensitivity score. The privacy label with a final score greater than 1 is used as the output of the first sub-network, completing the process of combining personal privacy preference modeling.

[0088] The original image, along with a privacy label annotated with individual privacy preferences and the image's scene context, is fed into a deep image prediction fusion network. The network then outputs a privacy label predicted based on the image's content, along with a prediction of whether the image is private. This method relies on pre-trained image feature extraction models and deep learning prediction models, eliminating the need for training the image feature extraction model. The trained model can directly predict the results of the present invention with high accuracy.

[0089] The present invention also proposes an image privacy prediction system based on scene context information perception, which is characterized by including a scene context information selection module when the image is released, an image input preprocessing module, a personal privacy preference setting module, a deep image cross-modal prediction network, and an image prediction result output module. The scene context information selection module inputs the scene context information when the image is released, the image input preprocessing module preprocesses the input original image to obtain a preprocessed image, and performs image content recognition, wherein the preprocessing method includes but is not limited to image blurring operation and pixelation operation; the deep image cross-modal prediction network performs feature fusion on the original image and the scene context information, and the image prediction result output module outputs the final prediction result.

[0090] The method described in this invention can be used in a system for predicting image privacy. Figure 3 As shown. Among them, Figure 3 A system process for image privacy prediction that combines personal privacy preferences with contextual information is demonstrated. First, the user sets their personal privacy preferences. The system captures the images and their contextual information that need to be shared and uploaded to social media. The captured image information is combined with the personal privacy preferences to perform image privacy prediction. Finally, the system determines whether the image is private and outputs the image's privacy label to remind the user.

[0091] This embodiment also provides a specific training process for the deep image anonymization network, which specifically includes the following steps:

[0092] 1) Dataset and preprocessing

[0093] VISPR dataset: Contains 22,000 images belonging to 28 privacy attributes, and each image is annotated with the privacy attributes it contains, such as whether it contains a face or a mobile phone number. The training set of this dataset is used to train the model in this embodiment, and the test set is used to test the model.

[0094] Use pre-trained image processing tools to detect, crop, and align the images in the above dataset, keeping the size and resolution of all images the same.

[0095] 2) Network training

[0096] The proposed deep image prediction network is trained using the VISPR training set. Three basic image feature extraction models are used in the training, and a total of five training results are produced. Among them, the three basic image feature extraction models are:

[0097] VGG network structure

[0098] Resnet network structure

[0099] MobileNetV2 network structure

[0100] After comparing the training results of the three basic models, the Resnet50 network structure of the present invention has the best effect. The feature extraction model of the image privacy label is:

[0101] word2vec

[0102] The feature extraction model for image privacy captions is:

[0103] sentence2vector

[0104] The present invention also proposes an image privacy prediction system based on scene context perception, such as Figure 3 The system is used to implement an image privacy prediction method based on scene context perception. The image shared by the user and the scene information generated by sharing the image are used as the input of the system. Combined with the user's customized privacy preferences, the system judges

[0105] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A scene context-aware image privacy prediction method, characterized in that: The specific steps include: Obtain the image to be shared, as well as the image's scene context information, namely, the time, location, and target audience of the image; Build a privacy label prediction network and use it to predict the privacy label of the image to be shared. The privacy label prediction network performs the following steps: Obtain the user's sensitivity to each tag. The more private the user thinks the tag is, the higher the sensitivity of the tag is. Use the Resnet network to extract image features from images shared by users, use the softmax classifier to perform multi-label classification on the image features, and obtain the confidence score of each label; Get the value of each label's confidence multiplied by the sensitivity level corresponding to the label, and use the label corresponding to the value greater than 1 as the privacy label corresponding to the image; A cross-modal image privacy prediction network based on scene context awareness is constructed. The image to be shared, the image's privacy label, and the image's scene context information are input into the network to predict whether the image is a private image. The process of determining whether an image is a private image based on scene context awareness includes: Based on the privacy label of the image and the scene context information of the image, an affinity matrix representing the correlation between the privacy label of the image and the scene context information of the image is obtained, including: A=(R a W a )(R d W d ) T Where A is the affinity matrix; R a The privacy tag is encoded by word2vec and then processed by the GRU model; R d After embedding the scene context information using the sentence2vector model, a vector with the same dimension as the privacy label is obtained; W α 、W d is the mapping matrix; The affinity matrix is used to extract the salient information of the privacy label from the privacy label of the image, and the salient information of the scene context information is extracted from the scene context information of the image; The private text features of the scene context information are integrated with the salient features of the privacy tag to obtain first private tag information; the private text features of the privacy tag are integrated with the salient features of the scene context information to obtain private feature information; Fusing the first privacy label information with the image features based on cross attention to obtain the second privacy label information; The similarity between the image feature and the second privacy tag information is used as the local similarity, the similarity between the image feature and the privacy feature information is used as the global similarity, and the weighted addition of the local similarity and the global similarity is used as the similarity between the image and the privacy information; The image is sorted according to the similarity between the image and the private information. If the similarity exceeds the set threshold, the image is judged to be a private image, otherwise it is not a private image.

2. The image privacy prediction method based on scene context awareness according to claim 1, characterized in that: Based on historical data, obtaining the sharing time, sharing location, and target audience corresponding to the predicted tag includes the following steps: Obtain the sharing time, sharing location, and target audience of the image when it is published; Set privacy labels, manually annotate historical data sets with single attribute labels, and calculate the privacy scores of historical data sets with single attribute labels based on sharing time, sharing location, and sharing target group; The highest score of each single-label dataset in terms of sharing time, sharing location, and sharing target population is taken as the sensitivity corresponding to the single label.

3. The image privacy prediction method based on scene context awareness according to claim 2, characterized in that: If N privacy labels are set, for the nth privacy label, the single label image corresponding to the label is collected as the single label attribute set corresponding to the label, and each image in the label is manually annotated. The sharing time, sharing location and sharing target group of each image are judged and scored. The score ranges from 0 to 4 points, where 0 indicates no violation, 1 indicates slight violation, 2 indicates violation, 3 indicates relatively serious violation of privacy, and 4 indicates very serious violation of privacy.

4. The image privacy prediction method based on scene context awareness according to claim 2 or 3, characterized in that: Sharing time includes working time and rest time, sharing locations include public places, formal places, and private places, and sharing target groups include strangers, general relationship groups, and close relationship groups.

5. The image privacy prediction method based on scene context awareness according to claim 1, characterized in that: The process of obtaining the first privacy tag information and privacy feature information includes: in, is the privacy label feature associated with the scene context information; g r is the number of privacy label types; is the privacy text feature associated with the privacy label; R a The vector obtained after word2vec encoding of the privacy label; R d After embedding the scene context information using the sentence2vector model, a vector with the same temperature as the privacy tag is obtained; E a Indicates the first privacy label information; E d Represents privacy feature information.

6. The image privacy prediction method based on scene context awareness according to claim 1, characterized in that: The process of obtaining the second privacy tag information includes: The first private label information after passing through three fully connected layers is mapped into a query vector through a trainable matrix; Map image features into key vectors and value vectors through two trainable matrices respectively; An attention value is calculated based on the query vector, the key vector, and the value vector, and the attention value is used as the second private label information.

7. The image privacy prediction method based on scene context awareness according to claim 1, characterized in that: The acquisition of the similarity between the image and the private information includes: S(I,M)=w1S local (I,M)+w2S glocal (I,M) S glocal (I,M)=cosine(Pool(E v ),Pool(E d )) Where S(I,M) represents the similarity between the image and the private information, I represents the image feature, and M represents the private information feature; w1 and w2 are balance factors, and w1+w2=1; sigmoid() represents the sigmoid activation layer; MLP() represents the two-layer perceptron; [|] represents the concatenation operation; Pool() represents the average pooling operation; E v Represents image features; E d Represents privacy feature information, Indicates the second privacy label information.

Images