A Multi-Dimensional Privacy Information Sensing and Collection Method

Abstract

This invention discloses a multi-dimensional privacy information perception and collection method. It constructs a formal description model of multi-dimensional privacy information, characterizing privacy information from multiple perspectives, including its composition, attributes, permissions, and control. Based on active learning and transfer learning algorithms, it builds a metadata intelligent perception, identification, and labeling method adaptable to various application scenarios and privacy protection strategies. This patent's advantages in adapting to multiple scenarios and information types allow it to adapt to the impact of changing application scenarios on privacy properties, reduce the concealment of privacy information caused by multiple information types, and improve the accuracy of privacy information perception. It focuses on perceiving multiple types of privacy information, comprehensively perceiving different privacy types such as text, images, and sound. Compared to existing single-type privacy perception methods, it is more beneficial for subsequent privacy strategy formulation and privacy control implementation, improving user privacy security. It has strong privacy perception capabilities, can process different types of data in parallel, and offers accurate privacy positioning and fast data processing.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method for multi-dimensional privacy information perception and collection. Background Technology

[0002] With the development of internet platforms, people have become accustomed to sharing their lives online through text, images, voice, and other media. However, this sharing behavior has brought about numerous privacy and security issues. The shared metadata may contain personal privacy information that the publisher is unaware of, as well as the privacy information of others. Users often do not verify or find it difficult to do so. Due to the inherent anonymity and subjectivity of privacy information, it may be published and disseminated without the sharer's knowledge, thus posing a significant risk of privacy breaches.

[0003] Privacy awareness and data collection technologies are fundamental to privacy protection. By accurately identifying and collecting user-generated private information, and then implementing tiered protection measures before sharing and dissemination, privacy leaks can be effectively reduced or even avoided. Currently, there is no comprehensive method for intelligently identifying, classifying, and tagging metadata that adapts to various application scenarios and privacy protection strategies for multi-dimensional information privacy awareness on the internet. Current privacy protection technologies largely focus on specific application scenarios and single data types, failing to meet the ever-growing demands for privacy protection, resulting in a persistently high risk of user privacy leaks.

[0004] In terms of text privacy feature extraction, existing methods lack discriminative power in extracting semantic features across different scenarios, resulting in poor privacy information classification or inaccurate privacy information localization. Regarding voice privacy feature extraction, existing methods primarily focus on privacy perception of user voiceprints and language habits, with insufficient extraction and identification of sensitive semantic content, or they only extract semantic content while ignoring the perception of user voiceprints and language habits, failing to achieve a balance between efficiency and high usability. In terms of image privacy feature extraction, most methods use commonly used SIFT and RGB features, which are insufficient for extracting deep semantic features, and the classification models have poor generalization ability. Summary of the Invention

[0005] To overcome the aforementioned problems in the existing technology, this invention proposes a multi-dimensional privacy information perception and collection method.

[0006] The technical solution adopted by this invention to solve its technical problem is: a multi-dimensional privacy information perception and collection method, comprising the following steps:

[0007] Step 1: Construct a multi-dimensional formal description model of privacy information, which includes multi-dimensional visual description labels.

[0008] Step 2: Collect the raw information M output by the user. Through intelligent perception algorithm, while acquiring the raw information M output by the user, record the underlying hardware environment information and software application environment information. Combine the privacy scenario, dissemination permission settings, and control requirements settings of the raw information M to improve some formal description tags. The raw information includes image information, voiceprint information, and text information.

[0009] Step 3: Apply different machine learning models to different types of raw information for privacy awareness, and record the information in the privacy attribute labels of the models described in Step 1;

[0010] Step 4: Using a usability / privacy assessment algorithm, the final usability / privacy assessment result is obtained through weighted calculation, and the result is added to the multi-dimensional visualization label of the model described in Step 1.

[0011] Step 3, which employs different machine learning models for privacy awareness of different types of original information, specifically includes: using an improved VGG19 model for privacy awareness of image information; using a BERT model for privacy awareness of text information; and using voiceprint information translated into text information via a speech translation model for privacy awareness.

[0012] The aforementioned multi-dimensional privacy information perception and collection method, specifically includes the following in step 1:

[0013] Step 1.1: Define the original information M as consisting of multiple privacy vectors I. Privacy vector I is the smallest, informative, indivisible, and non-overlapping atomic information in the original information.

[0014] Step 1.2: Define the multi-dimensional formal description label of privacy vector I.

[0015] In the above-mentioned multi-dimensional privacy information perception and collection method, the data type of privacy vector I in step 1.1 can be text data, voiceprint data, or image pixels; the privacy vector I has different privacy attributes depending on the amount of information it carries and different usage scenarios.

[0016] The above-mentioned multi-dimensional privacy information perception and collection method, wherein step 1.2 of the figurative description tag specifically includes: privacy scenario, which is an attribute describing the usage environment in which privacy information exists, and the attribute value is personal information, private domain information, public domain information, and underlying hardware information;

[0017] Privacy attributes are attributes that describe the specific type of privacy information. Attribute values ​​include non-privacy information, facial privacy, voiceprint privacy, and location information.

[0018] Privacy location refers to the position information attribute of privacy information component i in the original information M, which can quickly locate privacy information component i; the attribute value is paragraph number, frame number, and audio track number.

[0019] Dissemination permissions describe the restrictions on the dissemination of private information. By default, there are no dissemination restrictions, but the information producer can specify specific dissemination restrictions. These restrictions are divided into private information, semi-public information, and public information.

[0020] Control requirements describe the control and protection measures applied to the information. By default, it is in a copyable state, but the information producer can specify specific control restrictions, which are divided into visible only, copyable, and modifiable.

[0021] Availability measurement is a comprehensive measure of the availability of privacy information. It is derived from a comprehensive assessment of various attributes of the privacy information and provides a basis for judgment on possible subsequent privacy strategy selection. It is divided into high, medium, low, and unavailable.

[0022] Privacy measures are determined by a comprehensive assessment of the current attributes of privacy information, and are categorized as high, medium, low, and unprotected.

[0023] The above-mentioned multi-dimensional privacy information perception and collection method includes the following image information privacy perception process: S11, training of the improved VGG19 model. The specific training process includes: constructing a VGG19 convolutional layer and pre-training it using a publicly available large-scale image dataset to improve the model training efficiency; and using the adam optimizer to train and optimize the model based on the pre-trained parameters.

[0024] After the convolutional layer are the linear operation layer and the pooling layer. The linear operation layer can increase the feature representation ability of the model, while the pooling layer can reduce the model parameters and speed up the convergence speed of the model.

[0025] The privacy-awareness capability of the image was trained using a privacy-preserving image dataset, and the training results were tested using the privacy-preserving image dataset.

[0026] Image data can be processed through convolutional features generated by a model to obtain an attention distribution map. Points that are clustered in large numbers in the attention distribution map are the points where privacy features are concentrated in the image.

[0027] S12, Preprocess the image;

[0028] S13, the image data is divided into several minimum privacy vectors I. The privacy information carrying amount for the image data carrying privacy information is 1, and the privacy information carrying amount for other parts is 0. The privacy information carrying amount is recorded in the privacy attribute label.

[0029] In the aforementioned multi-dimensional privacy information perception and collection method, the voiceprint information is translated into text information using the FAIRSEQ model.

[0030] The above-mentioned multi-dimensional privacy information perception and collection method includes the following specific process for text information privacy perception: S21, BERT model training. The specific training process is as follows: first, the basic structure of the BERT model is constructed, then the Chinese corpus is pre-trained using a classification dataset, and the model training results are tested after the training is completed.

[0031] The BERT model uses a 12-layer Transformer structure, which is essentially an encoder-decoder structure. The encoder and decoder are composed of six identical basic layers stacked together, and each basic layer contains a fully connected feedforward neural network layer and a multi-head attention layer.

[0032] Multi-head attention mechanisms can calculate the attention weights at each position in the sentence during the encoding process; then, the implicit vector representation of the entire sentence is calculated in the form of a sum of weights.

[0033] S22, the input text information is segmented using a word segmentation tool, and the original text information is divided into several privacy vectors;

[0034] S23, the amount of privacy information carried in the text is set to 1, and the amount of privacy information carried in other parts is set to 0, and the amount of privacy information carried is recorded in the privacy attribute label.

[0035] The beneficial effects of the present invention are: (1) Compared with existing privacy perception technologies, this patent has the advantage of adapting to multiple scenarios and multiple information types, can adapt to the impact of changes in application scenarios on privacy, reduce the concealment of privacy information caused by multiple information types, and improve the accuracy of privacy information perception;

[0036] (2) This patent focuses on perceiving multiple types of privacy information, comprehensively perceiving different privacy types of text, images, and sounds. Compared with existing single-type privacy perception methods, it is more beneficial for the formulation of privacy strategies and the implementation of privacy control in the later stage, and improves user privacy security.

[0037] (3) This patent has strong privacy awareness capabilities, can process different types of data in parallel, and has accurate privacy positioning and fast data processing. Attached Figure Description

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] Figure 1 This is a flowchart of the privacy information perception and positioning process of the present invention;

[0040] Figure 2This is a schematic diagram of the formal description model of multi-dimensional privacy information of the present invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] This embodiment discloses a multi-dimensional privacy information perception and collection method, characterized by the following steps:

[0043] Step 1: Construct a multi-dimensional formal description model of privacy information;

[0044] Step 1.1: Define the original information M as consisting of multiple privacy vectors I. Privacy vector I is the smallest, informative, indivisible, and non-overlapping atomic information in the original information.

[0045] This model provides a formal description of atomic information from multiple dimensions. These "multi-dimensional" dimensions include the privacy attributes, access permissions, privacy positioning, dissemination permissions, and control requirements of atomic information.

[0046] The original information M is defined as consisting of multiple privacy vectors I, which are the smallest, informative, indivisible, and non-overlapping atomic information in the original information.

[0047] The data type of privacy vector I can be text data, voiceprint data, image pixels, etc.

[0048] Privacy vector I possesses different privacy attributes depending on the amount of information it carries and the different usage scenarios.

[0049] Step 1.2: Define the multi-dimensional formal description label of privacy vector I.

[0050] The specific descriptive tags include: privacy scenarios, which are attributes that describe the usage environment in which privacy information exists. Generally, attribute values ​​include personal information (such as notebooks, account books, and other private personal scenarios), private domain information (such as peer-to-peer communication, WeChat Moments, group chats, and other semi-public scenarios), public domain information (such as Weibo, Douyin, and other social scenarios involving strangers), and underlying hardware information.

[0051] Privacy attributes are attributes that describe the specific type of privacy information. Typical attribute values ​​include non-privacy information, facial privacy, voiceprint privacy, and location information.

[0052] Privacy positioning refers to the location information attribute of privacy information component i in the original information M, which can quickly locate privacy information component i; the attribute value is generally paragraph number, frame number, audio track number, etc.

[0053] Dissemination permissions describe the restrictions on the dissemination of private information. By default, there are no dissemination restrictions, but the information creator can specify specific dissemination restrictions. These are generally categorized as private information, semi-public information, and public information.

[0054] Control requirements describe the control and protection measures applied to this information. They default to a copyable state, but specific control restrictions can be specified by the information producer. They are generally categorized as visible only, copyable, and modifiable.

[0055] Availability metrics are comprehensive measures of the usability of privacy information. They are derived from a comprehensive assessment of various attributes of the privacy information and provide a basis for subsequent selection of possible privacy strategies. They are generally categorized as high, medium, low, and unusable.

[0056] Privacy measures are determined by comprehensively assessing various attributes of the privacy information. They are generally categorized as high, medium, low, or no protection.

[0057] Step 2: Collect the raw information M output by the user, including data types such as text, images, and audio. The number of privacy vectors I derived from the raw data M in the dataset is bounded. Most of these privacy vectors I carry very little privacy information, such as particles, interjections, noise, background pixels, and repeated frames. A small number of data carry highly privacy information and will be labeled and distinguished in subsequent steps. Using an intelligent perception algorithm, while acquiring the raw information M output by the user, the underlying hardware environment information and software application environment information are recorded. Combined with the privacy scenario, propagation permission settings, and control requirements of the raw information M, some formal descriptive labels are improved.

[0058] Step 3: Apply different machine learning models to different types of raw information for privacy awareness, and record the information in the privacy attribute labels of the models described in Step 1.

[0059] Different machine learning models are used for privacy awareness for different types of raw information. Specifically, the improved VGG19 model is used for privacy awareness of image information; the BERT model is used for privacy awareness of text information; and voiceprint information is translated into text information by a speech translation model and then used as text information for privacy awareness.

[0060] (1) The image information privacy awareness process specifically includes: S11, training the improved VGG19 model. The specific training process includes: constructing VGG19 convolutional layers and pre-training using the publicly available large-scale image dataset Imagnet to improve model training efficiency. Based on the pre-trained parameters, the adam optimizer is used to train and optimize the model;

[0061] After the convolutional layer are the linear operation layer and the pooling layer. The linear operation layer can increase the feature representation ability of the model, while the pooling layer can reduce the model parameters and speed up the convergence speed of the model.

[0062] We used the privacy-aware image dataset P3M-10k to train image privacy awareness capabilities and then used it to test the training results.

[0063] Image data can be processed through convolutional features generated by a model to obtain an attention distribution map. Points that are clustered in large numbers in the attention distribution map are the points where privacy features are concentrated in the image.

[0064] S12 preprocesses the image by dividing it into small blocks of 321*481 pixels each, which facilitates later processing.

[0065] S13, the image data is divided into several minimum privacy vectors I. The privacy information carrying amount for the image data carrying privacy information is 1, and the privacy information carrying amount for other parts is 0. The privacy information carrying amount is recorded in the privacy attribute label (for example, in a privacy image, the privacy information carrying amount for the pixel representing the face is 1; the privacy information carrying amount for the pixel representing the white background wall is 0).

[0066] (2) Voiceprint information is trained using the existing FAIRSEQ model, which can be used for speech-to-text modeling tasks, such as end-to-end speech recognition and speech-to-text translation. We translate speech information into text information, and then use a text privacy-aware model together with the text information to perceive and locate privacy information. This approach reduces the speech task to a text task, thereby reducing the difficulty of system operation and improving the system speed.

[0067] The main steps for building the FAIRSEQ model are as follows:

[0068] First, we build the FAIRSEQ model structure, as it is a mature open-source speech translation model. Therefore, there is no need to modify it; the existing settings can be applied directly.

[0069] The model was then trained using the open-source speech translation dataset VoxPopuli, and the training results were tested after training. Once a high translation accuracy was achieved, the model was packaged and used to output the translated text, which was then passed to a text privacy-aware model for privacy information detection and localization.

[0070] (3) The specific process of text information privacy awareness includes: S21, BERT model training. The specific training process is as follows: First, the basic structure of the BERT model is constructed, then the Chinese corpus is pre-trained using the classification dataset, and the model training results are tested after the training is completed.

[0071] The BERT model uses a 12-layer Transformer structure, which is essentially an encoder-decoder structure. The encoder and decoder are composed of six identical basic layers stacked together, and each basic layer contains a fully connected feedforward neural network layer and a multi-head attention layer.

[0072] Multi-head attention mechanisms can calculate the attention weights at each position in the sentence during the encoding process; then, the implicit vector representation of the entire sentence is calculated in the form of a sum of weights; using the attention weights calculated by the model, text privacy is intelligently perceived, and a judgment threshold is reasonably set. The position where the attention weight is higher than the judgment threshold is the location where privacy information exists in the text information.

[0073] S22, the input text information is segmented using a word segmentation tool, and the original text information is divided into several privacy vectors;

[0074] S23, the amount of privacy information carried in the text is set to 1, and the amount of privacy information carried in other parts is set to 0, and the amount of privacy information carried is recorded in the privacy attribute label.

[0075] Step 4: Using a usability / privacy assessment algorithm, the final usability / privacy assessment result is obtained through weighted calculation, and the result is added to the multi-dimensional visualization description label of the model described in Step 1.

[0076] The usability / privacy assessment algorithm formally describes tag information using multi-dimensional privacy information in a privacy vector. Based on the different privacy weights of each dimension, the algorithm first determines the privacy context in which the privacy information resides. Under different privacy contexts, the privacy weight of the same tag will change accordingly. The final usability / privacy assessment result is obtained through weighted calculation.

[0077] Availability metrics are generally categorized as high, medium, low, and unavailable. Privacy metrics are generally categorized as high, medium, low, and unprotected. These metrics can serve as an important basis for selecting multi-level privacy protection strategies, allowing users to make adaptive strategy choices based on the availability / privacy metrics of privacy information.

[0078] The usability / privacy assessment algorithm comprehensively evaluates the usability and privacy metrics of privacy information, and uses the results to further supplement the multi-dimensional formal description labels of privacy information in the privacy vector.

[0079] The usability / privacy assessment algorithm assigns a weight to each attribute of privacy information, and this weight varies depending on the privacy scenario. For example, the facial privacy weight of a piece of privacy information is 0.9 in a private information scenario (such as WeChat Moments), but becomes 1.2 in a public information scenario (such as TikTok). Each piece of privacy information has a different weight in different privacy scenarios. The usability / privacy assessment algorithm performs a weighted calculation and uses the final calculated score to classify the usability / privacy measure of the privacy information.

[0080] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its spirit and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A method for multi-dimensional privacy information perception and collection, characterized in that, Includes the following steps: Step 1: Construct a multi-dimensional formal description model of privacy information, which includes multi-dimensional visual description labels. Step 2: Collect the raw information M output by the user. Through intelligent perception algorithm, while acquiring the raw information M output by the user, record the underlying hardware environment information and software application environment information. Combine the privacy scenario, dissemination permission settings, and control requirements settings of the raw information M to improve some formal description tags. The raw information includes image information, voiceprint information, and text information. Step 3: Apply different machine learning models to different types of raw information for privacy awareness, and record the information in the privacy attribute labels of the models described in Step 1; Step 4: Using a usability / privacy assessment algorithm, the final usability / privacy assessment result is obtained through weighted calculation, and the result is added to the multi-dimensional visualization label of the model described in Step 1. Step 3, which employs different machine learning models for privacy awareness of different types of original information, specifically includes: using an improved VGG19 model for privacy awareness of image information; using a BERT model for privacy awareness of text information; and using voiceprint information translated into text information via a speech translation model for privacy awareness.

2. The multi-dimensional privacy information perception and collection method according to claim 1, characterized in that, The model construction in step 1 specifically includes: Step 1.1: Define the original information M as consisting of multiple privacy vectors I. Privacy vector I is the smallest, informative, indivisible, and non-overlapping atomic information in the original information. Step 1.2: Define the multi-dimensional formal description label of privacy vector I.

3. The multi-dimensional privacy information perception and collection method according to claim 2, characterized in that, In step 1.1, the data type of privacy vector I can be text data, voiceprint data, or image pixels; the privacy vector I has different privacy attributes depending on the amount of information it carries and different usage scenarios.

4. The multi-dimensional privacy information perception and collection method according to claim 2, characterized in that, The visual description of the tag in step 1.2 specifically includes: privacy scenario, which is an attribute that describes the usage environment in which privacy information exists, and the attribute value is personal information, private domain information, public domain information, and underlying hardware information; Privacy attributes are attributes that describe the specific type of privacy information. Attribute values ​​include non-privacy information, facial privacy, voiceprint privacy, and location information. Privacy location refers to the location information attribute of privacy information component i in the original information M, which can quickly locate privacy information component i; the attribute value is paragraph number, frame number, and audio track number. Dissemination permissions describe the restrictions on the dissemination of private information. By default, there are no dissemination restrictions, but the information producer can specify specific dissemination restrictions. These restrictions are divided into private information, semi-public information, and public information. Control requirements describe the control and protection measures applied to the information. By default, it is in a copyable state, but the information producer can specify specific control restrictions, which are divided into visible only, copyable, and modifiable. Availability measurement is a comprehensive measure of the availability of privacy information. It is derived from a comprehensive assessment of various attributes of the privacy information and provides a basis for judgment on possible subsequent privacy strategy selection. It is divided into high, medium, low, and unavailable. Privacy measures are determined by a comprehensive assessment of the current attributes of privacy information, and are categorized as high, medium, low, and unprotected.

5. The multi-dimensional privacy information perception and collection method according to claim 1, characterized in that, The image information privacy awareness process specifically includes: S11, training the improved VGG19 model. The specific training process includes: constructing VGG19 convolutional layers and pre-training using publicly available large-scale image datasets to improve model training efficiency; and using the adam optimizer to train and optimize the model based on the pre-trained parameters. After the convolutional layer are the linear operation layer and the pooling layer. The linear operation layer can increase the feature representation ability of the model, while the pooling layer can reduce the model parameters and speed up the convergence speed of the model. The privacy-awareness capability of the image was trained using a privacy-preserving image dataset, and the training results were tested using the privacy-preserving image dataset. Image data can be processed through convolutional features generated by a model to obtain an attention distribution map. Points that are clustered in large numbers in the attention distribution map are the points where privacy features are concentrated in the image. S12, Preprocess the image; S13, the image data is divided into several minimum privacy vectors I. The privacy information carrying amount for the image data carrying privacy information is 1, and the privacy information carrying amount for other parts is 0. The privacy information carrying amount is recorded in the privacy attribute label.

6. The multi-dimensional privacy information perception and collection method according to claim 1, characterized in that, The voiceprint information is translated into text information using the FAIRSEQ model.

7. The multi-dimensional privacy information perception and collection method according to claim 1, characterized in that, The specific process of privacy awareness of text information includes: S21, BERT model training. The specific training process is as follows: first, the basic structure of the BERT model is constructed, then the Chinese corpus is pre-trained using a classification dataset, and the model training results are tested after the training is completed. The BERT model uses a 12-layer Transformer structure, which is essentially an encoder-decoder structure. The encoder and decoder are composed of six identical basic layers stacked together, and each basic layer contains a fully connected feedforward neural network layer and a multi-head attention layer. Multi-head attention mechanisms can calculate the attention weights at each position in the sentence during the encoding process; then, the implicit vector representation of the entire sentence is calculated in the form of a sum of weights. S22, the input text information is segmented using a word segmentation tool, and the original text information is divided into several privacy vectors; S23, the amount of privacy information carried in the text is set to 1, and the amount of privacy information carried in other parts is set to 0, and the amount of privacy information carried is recorded in the privacy attribute label.

Images