Mobile application privacy policy compliance detection method based on pre-training model

Abstract

The invention discloses a mobile application privacy policy compliance detection method based on a pre-training model, and aims to realize efficient and intelligent privacy policy compliance automatic detection. The method comprises the following steps: firstly, constructing a hierarchical privacy policy compliance detection index system according to domestic related laws and regulations and standards; secondly, collecting an original text of the privacy policy through a web crawler technology, and constructing an unlabeled corpus after cleaning and structured processing; thirdly, constructing a multi-label classification data set based on a mode of combining a large language model and manual review; mapping the text and the label to a unified semantic vector space by adopting a text-label joint embedding strategy, and inputting the text and the label into a multi-granularity classification model; according to the model, on the basis of an ERNIE pre-training model, context feature enhancement and deep semantic interaction are realized through a bidirectional long-short-term memory network, a self-attention mechanism and a text-label cross attention mechanism, so that the multi-label classification performance is remarkably improved; finally, according to a label prediction result output by the model and a preset index system, compliance judgment is automatically completed, and a structured detection report is generated. According to the invention, the automation degree and efficiency of privacy policy compliance detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing technology and privacy policy compliance detection, specifically to a method for detecting the compliance of mobile application privacy policies based on a pre-trained model. Background Technology

[0002] With the rapid development of internet technology, mobile internet applications have become the core carrier of digital services, deeply penetrating all dimensions of social production and life. However, the widespread use of mobile applications has also led to a continuous increase in the risk of personal information privacy leaks: many applications engage in illegal activities such as collecting personal information without users' explicit consent and excessively requesting permissions, seriously infringing on users' privacy rights and making privacy protection increasingly urgent. To regulate industry order, my country has established a legal standard system centered on the "Personal Information Protection Law of the People's Republic of China," which clearly requires mobile applications to fully inform users of their personal information processing rules through privacy policies. The compliance of privacy policies has become a key link in protecting user rights and network data security.

[0003] Despite the continuous improvement of the legal framework, non-compliance with privacy policies remains widespread. The Ministry of Industry and Information Technology's spot checks revealed that many applications and software development kits violated relevant laws and regulations due to issues such as "illegally collecting personal information" and "forcibly requesting unnecessary permissions." This reflects significant shortcomings in the efficiency, accuracy, and scalability of current privacy policy compliance testing. Existing testing methods largely rely on manual review or simple keyword matching, which suffers from strong subjectivity, insufficient semantic understanding, and limited coverage, making it difficult to meet the compliance testing needs of the massive number of mobile applications.

[0004] Based on this, the present invention aims to utilize the powerful natural language understanding capabilities of pre-trained models to construct an automated, high-precision, and scalable method for detecting privacy policy compliance. Through technologies such as multi-label classification and text-label joint embedding, it achieves intelligent compliance assessment of privacy policy texts, thereby effectively addressing the shortcomings of existing technologies. Summary of the Invention

[0005] A method for mobile application privacy policy compliance detection based on a pre-trained model, characterized by the following steps:

[0006] S1: Based on the "Guidelines for Self-Assessment of Illegal and Irregular Collection and Use of Personal Information by Apps", the "Personal Information Protection Law of the People's Republic of China", and the "Information Security Technology - Personal Information Security Specification" (GB / T 35273-2020), a hierarchical compliance testing system is constructed, including primary and secondary indicators. The primary indicators include: basic information A1, personal information collection rules A2, personal information storage A3, personal information protection A4, user rights protection mechanism A5, and personal information sharing, transfer, and public disclosure A6. Each primary indicator has several detectable secondary indicators. For example, basic information includes company name B1, registered address B2, contact information B3, and publication, effective, or update date B4.

[0007] S2: Obtain the raw text of the privacy policy from the app store using web crawling technology, clean and format it, and divide it into blocks according to paragraph structure, ensuring that each block does not exceed the maximum input length of the pre-trained model, thereby constructing the initial unlabeled corpus, including the following steps:

[0008] S2-1: Using the Scrapy web crawler framework combined with the Selenium automated testing tool, targeted crawling of the privacy policy text of mobile applications in app stores;

[0009] S2-2: Clean the crawled raw text and segment it into sentences according to semantic logic and paragraph structure to ensure that each block of content does not exceed the maximum input length of the pre-trained model, thus obtaining structured sentence-level text units;

[0010] S3: Based on the secondary detection indicators defined in S1, structured prompt words are designed, and the prompt word engineering guides the large language model to perform multi-label pre-annotation on sentence-level text units; for conflicting and ambiguous sentences in the pre-annotation results, a manual review and correction mechanism is introduced, and finally a high-quality privacy policy text unit-compliance label multi-label classification dataset is constructed.

[0011] S4: Employing a text-tag joint embedding method, compliant tags are mapped to special tokens and input together with the text sequence into a pre-trained model to generate a deep feature vector that integrates the semantic association between text and tags. This includes the following steps:

[0012] S4-1: Generate a unique token identifier for each compliance inspection label, in the form of... Furthermore, all tag tokens are expanded into the vocabulary of the pre-trained model to achieve independent embedding representations of tags;

[0013] S4-2: For each text unit and corresponding label set in the training sample, construct the joint input sequence according to the following structure: ;

[0014] S4-3: The joint sequence is tokenized using the ERNIE-3.0-base-zh tokenizer and then input into the embedding and encoding layers of the model. A unified feature representation fusing textual semantics and label information is extracted through the self-attention mechanism of the pre-trained model. The final vector of the joint sequence is as follows: .

[0015] S5: Transform the privacy policy compliance detection problem into a multi-label classification task, construct a multi-label classification model based on text-label joint embedding with multiple granularities, and finally perform end-to-end training using the binary cross-entropy loss function to achieve compliance label prediction for privacy policy texts, including the following steps:

[0016] S5-1: Extracting text vector sequences based on the joint sequence vector of S4-3 The input is fed into a bidirectional long short-term memory network and a text self-attention module for context modeling, resulting in text feature representations. The feature is mapped to the label space through a linear layer, and supervised training is performed using the binary cross-entropy loss function to obtain the text-granular classification loss. ;

[0017] S5-2: Based on text vectors and label vector It uses a regularized embedding layer and a text-label cross-attention mechanism to interact and model, extracting fused features. Furthermore, it maps to the label system through a linear layer and calculates the text-label interaction loss. ;

[0018] S5-3: The overall training objective of the model is a weighted combination loss. The parameters of all modules are jointly optimized through end-to-end training. Commonly used Micro / Macro-F1, Recall, and Precision are selected as model performance evaluation metrics. The optimal model is selected based on the highest Micro-F1 score on the validation set for subsequent inference tasks.

[0019] S6: Based on the compliance detection index system constructed in S1, and combined with the label prediction results output by the S5-3 model inference, verify the compliance of the privacy policy, including the following steps:

[0020] S6-1: Using the data cleaning and semantic segmentation methods described in step 2-2, preprocess the privacy policy text to be detected to generate structured sentence-level text units; input these units into the training model selected in S5-4 to infer the predicted tag set corresponding to the privacy policy. ;

[0021] S6-2: Predict the label set With the set of pre-defined compliance detection indicators constructed by S1 Perform a comparison; if Incomplete tag set If all required labels are not present, the privacy policy is deemed non-compliant; if... Full coverage If the result is satisfactory, the privacy policy is deemed to comply with compliance requirements, and a privacy compliance report is issued accordingly.

[0022] The beneficial effects of this invention are as follows:

[0023] (1) A hierarchical and quantifiable compliance testing indicator system has been constructed. Based on domestic laws, regulations and standards such as the "Guidelines for Self-Assessment of Illegal and Irregular Collection and Use of Personal Information by Apps", the "Personal Information Protection Law of the People's Republic of China", and the "Information Security Technology Personal Information Security Specification" (GB / T 35273-2020), a testing system has been systematically constructed, covering multiple primary indicators and their subdivided secondary indicators, including the basic information of privacy policies, data collection rules, data storage and processing, protection of the rights of the subject, and data sharing, transfer and public disclosure. This provides a structured and operable standard basis for compliance judgment.

[0024] (2) A multi-label classification model with joint embedding and multiple granularities is proposed. By introducing a text-label joint embedding mechanism, combined with a bidirectional long short-term memory network, a self-attention mechanism and a text-label cross-attention module, a deep semantic parsing and label association modeling of privacy policy text is realized, which effectively improves the model's ability to identify complex compliance requirements and its classification accuracy.

[0025] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0027] Figure 1 This is a flowchart of the method in a specific embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a multi-label classification model with joint embedding at multiple granularities;

[0029] Figure 3This is a schematic diagram of a bidirectional long short-term memory network combined with a text self-attention submodule;

[0030] Figure 4 A schematic diagram of the text-label cross-attention submodule;

[0031] Figure 5 This is a flowchart illustrating the abstract of the present invention. Detailed Implementation

[0032] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. The technical solution of the present invention will be further described below in conjunction with specific embodiments and accompanying drawings.

[0033] This invention proposes a method for detecting the compliance of mobile application privacy policies based on a pre-trained model. Please refer to [link / reference]. Figure 1 As shown, the specific method is explained below:

[0034] Step 1: Based on the "Self-Assessment Guidelines for Illegal and Irregular Collection and Use of Personal Information by Apps," the "Personal Information Protection Law of the People's Republic of China," and the "Information Security Technology - Personal Information Security Specification" (GB / T 35273-2020), a hierarchical compliance testing system is constructed, including primary and secondary indicators. The primary indicators include: Basic Information A1, Personal Information Collection Rules A2, Personal Information Storage A3, Personal Information Protection A4, User Rights Protection Mechanism A5, and Personal Information Sharing, Transfer, and Public Disclosure A6. Each primary indicator has several detectable secondary indicators. For example, basic information includes company name B1, registered address B2, contact information B3, and publication, effective, or update date B4. The indicator system for compliance with privacy policies is shown in Table 1.

[0035] Table 1 Privacy Policy Compliance Indicator System

[0036] Primary indicators Secondary indicators Indicator basis A1 Basic Information B1 Company Name Guidelines 9, Individual Income Tax Law 23 B2 Registration Address Guidelines 9, Standards 5.5 B3 Contact Information Guidelines 9, Individual Income Tax Law 52 B4 Release, effective, or updated date Guide 17 A2 Personal Information Collection Rules B5 Purpose and Scope of Collection Guidelines 21 B6 Cookie and similar technologies are used Guidelines 21 B7 Notification and Consent Mechanism Guidelines 23 B8 Personal Information Outbound Status Guidelines 12, Individual Income Tax Protection Law 38 A3 Personal Information Storage B9 Storage Region Guidelines 10 B10 Storage Period Guidelines 10, Standards 5.5 B11 Overdue processing method Guidelines 10 A4 Personal Information Protection B12 Security Mechanism Guidelines 13 B13 Protection of Minors' Personal Information Individual Protection Law, Article 31 A5 User Rights Protection Mechanism B14 Personal Information Inquiry Guidelines 15, 31 B15 Personal Information Correction Guidelines 15, 31 B16 Deletion of Personal Information Guidelines 15, 31 B17 User Account Cancellation Guidelines 15, 30 B18 Withdrawal of previously agreed-upon authorization Guidelines 15 A6. Status of Personal Information Sharing, Transfer, and Public Disclosure B19 Define the scope, purpose, and timeframe. Guide 14 B20 Recipient type or identity Guide 14

[0037] It should be noted that the above-mentioned secondary indicators have a clear mapping relationship with relevant laws and regulations. Taking the secondary indicator "B1 Company Name" as an example, it is based on Article 23 of the "Personal Information Protection Law of the People's Republic of China," which stipulates that when a personal information processor provides personal information to other processors, it shall inform the individual of the recipient's name or other information.

[0038] Step 2: Based on the compliance detection index system built in Step 1, the original text of the privacy policy is obtained from the app store using web crawling technology. This text is then cleaned and formatted, and divided into blocks according to paragraph structure, ensuring that each block does not exceed the maximum input length of the pre-trained model. This forms the initial unlabeled corpus, as detailed below:

[0039] Step 2-1: Use the Scrapy web crawling framework in conjunction with the Selenium automated testing tool to perform targeted crawling of the privacy policy text of mobile applications in app stores;

[0040] It should be noted that the specific criteria for this data crawling were for three categories of high-frequency mobile applications: utility apps, social apps, and e-commerce apps. The system crawled the privacy policy texts of 50 apps in each category, totaling over 150 apps. The data collection period was limited to publicly available versions from January 1, 2024 to June 30, 2025. During the data preprocessing stage, duplicate policies from the same app were removed by URL deduplication, and strict text validity checks were performed to exclude invalid text such as pure images or garbled characters, ensuring that each policy retained had at least 500 valid characters. After the above filtering process, a raw text dataset containing 150 valid privacy policies was finally constructed.

[0041] Step 2-2: Clean the raw text that has been crawled and segment it into sentences according to semantic logic to obtain structured sentence-level text units;

[0042] It should be noted that the data cleaning process includes removing HTML tags, special characters, and meaningless blank lines, and uniformly converting the text encoding to UTF-8 format. In the segmentation stage, natural paragraphs are used as the main processing unit, and combined with semantic boundary markers such as Chinese periods and semicolons, the text is further segmented into sentence-level units with complete semantics, and finally a structured sentence-level unlabeled dataset is constructed.

[0043] Step 3: Based on the secondary detection indicators identified in Step 1, design structured prompt words, and use prompt word engineering to guide the large language model to perform multi-label pre-annotation on sentence-level text units; for conflicting and ambiguous sentences in the pre-annotation results, introduce a manual review and correction mechanism, and finally construct a high-quality privacy policy text unit-compliance label multi-label classification dataset.

[0044] It should be noted that the final generated single-line labeled data is in JSON format, as shown in the following example:

[0045] {"text": "a. When you register and log in to Doubao and related services, you can create an account and complete the registration using your mobile phone number. The mobile phone number is necessary information to comply with national laws and regulations regarding real-name registration on the Internet. If you do not provide a mobile phone number for registration and login, we may not be able to provide you with functions such as AI dialogue.", "label": [5,7]}; where 5 corresponds to "B5 Purpose and Scope of Collection" and 7 corresponds to "B7 Notification and Consent Mechanism".

[0046] Step 4: Employ a text-tag joint embedding method to map compliant tags to special tokens and input them together with the text sequence into the pre-trained model, generating a deep feature vector that integrates the semantic association between the text and the tags, as detailed below:

[0047] Step 4-1: Generate a unique token identifier for each compliance inspection label, in the form of... Furthermore, all tag tokens are expanded into the vocabulary of the pre-trained model to achieve independent embedding representations of tags;

[0048] Step 4-2: For any training sample in the dataset ,in The text content of a single labeled data entry is split into: Etc. sub-fragments; A sequence of labels consisting of compliant labels, containing Tags; generate a joint sequence according to the following fixed structure: ,in Sentence-level feature identifiers Used as a separator between text and tags;

[0049] Step 4-3: Using ERNIE3.0-base-zh as a pre-trained model, its tokenizer is used to tokenize the constructed joint sequence, and the token sequence is mapped to the corresponding word embedding vector set through the model's embedding layer: ,in The word embedding vector generated after the corresponding token is mapped through the embedding layer.

[0050] Step 5: Model the privacy policy compliance detection problem as a multi-label classification task, employing a multi-label classification model based on text-label joint embedding with multiple granularities, according to... Figure 2 The model architecture shown and the parameter configuration listed in Table 2 were used to complete model training and inference in an environment of Python 3.12, PyTorch 2.8.0+cu129, and Transformers 4.57.0, outputting compliance labels corresponding to the privacy policy text. The specific implementation is as follows:

[0051] Table 2 List of Experimental Parameters

[0052] Parameter name Parameter value BATCH_SIZE 16 SEQ_LENGTH 300 LEARNING_RATE 2e-5 EPOCHS 20 ALPHA 0.5 THRESHOLD 0.5

[0053] Step 5-1: Based on the word embedding vector set obtained in Step 4-3, extract the text-level vectors: Using this as input, through Figure 3 The model shown is trained. This model includes a BiLSTM layer, an Attention layer, and a Linear layer; As input to the BiLSTM, obtain the bidirectional hidden layer state vector containing statement context information. Its calculation formula is ;by As input to the Attention layer, key features are aggregated through the attention mechanism to ultimately obtain the feature vector. Its calculation formula is ,in The weight information is represented by the following formula: ,in The representative weight parameter is calculated using the following formula: ; Eigenvector After processing by a linear layer, the output value is compressed to the [0,1] interval by activation with a sigmoid function. Then, by comparing with a preset threshold, the binary cross-entropy loss (BCE loss) is selected, denoted as […]. The training process for multi-label classification is completed.

[0054] It should be noted that in the BiLSTM layer, where This represents the hidden state of the feedforward LSTM cell at time t. Let be the hidden state of the inverse LSTM unit at time t; in the Attention layer, This represents a trainable weight vector. express The transpose of , Represents the weights in the attention mechanism, initialized using the default Kaiming uniform initialization. This indicates the bias term.

[0055] Step 5-2: Based on the word embedding vector set obtained in Step 4-3, extract text-level vectors. Recorded as and label-level vectors Recorded as ,use Figure 4 The model shown is trained. This model includes a regularized embedding layer, a text-label attention layer, and a linear layer; firstly, the model is trained... and conduct Regularization yields a standardized text representation, calculated using the following formula: , ,in Definition of regularization ,and , Where N is the length of the text sequence, M is the length of the label sequence, and H is the dimension of the feature representation vector; text-label attention features are obtained through dot product operation. , Through the activation function Two-dimensional convolutional neural networks Extracting deep interaction information , combined and The function calculates the importance score of each word for the label prediction. Importance score and Multiply, then pass The function obtains the text-label feature vector. ; After processing by the linear layer, a binary cross-entropy loss optimization model is used, with the loss denoted as... The training process for multi-label classification is completed.

[0056] Step 5-3: The result obtained in Step 5-1 The result obtained in step 5-2 Parallel training is performed, and both methods use the binary cross-entropy loss function, the calculation formula of which is as follows: Where N is the total number of samples, Let be the total number of labels in a multi-label text classification system; the total loss function for training is the weighted sum of the two, and its calculation formula is: ,in These are hyperparameters used to balance the weight loss of the two sub-models. Commonly used micro / macro-F1 scores, micro / macro-recall, and micro / macro-precision are selected as model performance evaluation metrics. In actual training, the highest Micro-F1 score on the validation set is used as the model selection criterion, choosing the model with the best performance.

[0057] Step 6: Based on the compliance testing indicator system constructed above, and combined with the label prediction results output by the model, verify the compliance of the privacy policy and determine whether it meets all compliance requirements, as follows:

[0058] Step 6-1: Following the preprocessing method in Step 2-2, perform data cleaning and semantic segmentation on the privacy policy text to be detected to obtain structured sentence-level text units; input these text units into the optimal model selected in Step 5-3 for inference, and output the set of predicted tags corresponding to the privacy policy. ;

[0059] Step 6-2: For the actual set of tags With the preset set of compliance testing indicators Conduct compliance verification, if Incomplete tag set If the privacy policy is deemed non-compliant, then it is determined that the privacy policy does not meet compliance requirements; if... Completely cover the set of tags If the privacy policy is deemed compliant, a compliance document will be generated.

Claims

1. A method for detecting compliance of a mobile application privacy policy based on a pre-trained model, the method comprising: receiving a mobile application privacy policy; receiving a pre-trained model; and determining whether the mobile application privacy policy complies with the pre-trained model. The method comprises the following steps: S1: The system analyzes applicable laws, regulations and standards, and constructs a privacy policy detection index system containing multiple compliance requirements; S2: Obtain the original text of the privacy policy from the application store through web crawler technology, perform data cleaning and format normalization, and perform block processing according to the paragraph structure to ensure that each block does not exceed the maximum input length of the pre-trained model, thereby constructing an initial unlabeled corpus; S3: Based on the detection index system of S1 and the unlabeled corpus of S2, pre-labeling is performed using a large language model, and the pre-labeling results are reviewed and corrected by a domain expert to construct a multi-label classification data set with compliance labels; S4: A text-label joint embedding method is used to map compliance labels to special tokens and input them into a pre-trained model together with text sequences to generate deep feature vectors that integrate text and label semantic associations; S5: Convert the privacy policy compliance detection problem into a multi-label classification task, construct a multi-granularity multi-label classification model based on text-label joint embedding, and finally train end-to-end through a binary cross-entropy loss function to realize label prediction of the privacy policy text; S6: Analyze the label prediction probability output by S5, and according to the preset judgment threshold and the compliance detection index system constructed in S1, quantitatively evaluate the privacy policy text, determine its compliance with each compliance requirement one by one, and generate a structured compliance detection report. 2.The pre-trained model based mobile application privacy policy compliance detection method of claim 1, wherein, In step S1, the hierarchical privacy policy compliance detection index system is constructed based on the App Illegal and Irregular Collection and Use of Personal Information Self-Assessment Guide, the Personal Information Protection Law of the People's Republic of China, and the Information Security Technology Personal Information Security Specification (GB / T 35273-2020); the system includes multiple interrelated first compliance domains and detailed second detection items, wherein: the first compliance domain includes: basic situation, personal information collection rules, personal information storage, personal information protection, user right protection mechanism, personal information sharing, transfer, and public disclosure; each first compliance domain has quantifiable second detection items, for example, the basic situation has corresponding second detection items including company name, registered address, contact information, and publication, effective or update date.

3. The pre-trained model based mobile application privacy policy compliance detection method of claim 1, wherein, In step S3, based on the index system constructed in S1, a structured prompt word is designed, and the reasoning ability of a large language model is used to pre-label the sentence-level text units with multiple labels; For ambiguities and conflicts in the pre-labeling results, an artificial review and correction mechanism is introduced to finally construct a high-quality privacy policy text unit-compliance label multi-label classification data set. 4.The pre-trained model based mobile application privacy policy compliance detection method of claim 1, wherein, In steps S4 and S5, a multi-granularity classification model based on text-label joint embedding is used to complete the multi-label classification task; the model first maps text sequences and compliance label Token sequences to unified vector representations through a joint embedding layer; The joint vector is then encoded by the ERNIE pre-training model as a context encoder to capture basic semantic features; then the sequence context modeling capability is further enhanced through a bidirectional long short-term memory network and a self-attention mechanism to capture local correlation features of the text and the label; meanwhile, the deep interaction between the text semantics and the label definition is realized through a text-label cross-attention mechanism, the weight distribution between features is calculated, and end-to-end training is carried out based on a binary cross-entropy loss to realize the compliance label prediction of the privacy policy text.

Images