A method for dynamic measurement modeling of user privacy based on LSTM+CRF

Through the LSTM+CRF model and Scrapy crawler technology, the problem of obtaining user privacy and interest behavior characteristics in the existing technology is solved, efficient extraction of user sensitive features and scientific disclosure of dynamic changes trends, and the accuracy of knowledge graph construction is improved Spend.

CN114707062BActive Publication Date: 2024-12-06GUANGZHOU FUBAOPING TECHNOLOGY CO LTD
View PDF 16 Cites 0 Cited by

Patent Information

Application Number
CN202210277708.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-21
Publication Date
2024-12-06
Estimated Expiration
2042-03-21

AI Technical Summary

Technical Problem

The prior art is difficult to efficiently and accurately obtain user structured information from the data disclosed on the network, especially the user's privacy and interest behavior characteristics, resulting in insufficient accuracy and efficiency of user privacy dynamic metric modeling.

Method used

The user privacy dynamic metric modeling method based on LSTM+CRF is adopted, and user information is obtained through Scrapy crawlers, and the LSTM+CRF model is used for iterative training, which extracts the user's sensitive features, provides data for the knowledge graph, and optimizes the model training accuracy.

Benefits of technology

It realizes efficient extraction of user-sensitive features and scientific disclosure of dynamic changes trends, improving the accuracy of knowledge graph mapping and the effectiveness of user privacy protection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114707062B_ABST
    Figure CN114707062B_ABST
Patent Text Reader

Abstract

This invention discloses a method for dynamic measurement modeling of user privacy based on LSTM+CRF, belonging to the field of dynamic measurement technology of user privacy. The method includes the following steps: Step 1: Dynamic sensitive feature training text extraction. S1: Design an automatic crawler program based on the Scrapy framework. S2: Crawl the text input by the user from mobile application services, and simultaneously obtain and compare privacy-related text annotation information from a trusted third-party library. This invention utilizes Scrapy crawler to obtain user information from the network, and simultaneously obtains manually annotated privacy information from a trusted third party. Then, it trains the obtained information using an LSTM+CRF model to construct a dynamic privacy measurement model. The model iteratively extracts sensitive features of the user, providing data for knowledge graph construction, thereby optimizing the accuracy of the training model and scientifically revealing the dynamic changing trends of user sensitive features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of user privacy dynamic measurement, and specifically to a method for user privacy dynamic measurement modeling based on LSTM+CRF. Background Art

[0002] In recent years, with the breakthrough progress of deep learning theory and technology, more and more mobile intelligent applications have adopted deep learning models to quickly calculate, analyze and model massive amounts of user data, thereby providing decision support for various intelligent applications. For example, in the field of online shopping recommendations, by analyzing the user's browsing behavior on shopping websites and the click operations on products, the user's interest preference for this type of product can be obtained, thereby improving the recommendation efficiency, such as: recommending products browsed by users. In the field of advertising, by analyzing the user's operational behavior on various advertisements, the user's interest direction and user's future needs can be obtained and used to improve the accuracy of advertising, such as: various advertisements. The interest behavior characteristics of network users are inseparable from the extraction of user behavior entity relationships. Entity relationship extraction can extract the interest behavior characteristics of network users from a large amount of network text. With the development of the Internet, users publish more and more information on the Internet and involve various fields, which is disorganized. How to efficiently and accurately obtain useful user information from the public data on the Internet in a structured way has become an important issue facing scientific researchers.

[0003] To this end, we propose a method for modeling user privacy dynamic measurement based on LSTM+CRF. Summary of the invention

[0004] In view of the above problems and / or the existing problems in the method of user privacy dynamic measurement modeling based on LSTM+CRF, the present invention is proposed.

[0005] Therefore, the purpose of the present invention is to provide a method for dynamic measurement modeling of user privacy based on LSTM+CRF. By using the Scrapy crawler to obtain user information on the network, and then training the model based on the LSTM+CRF model, the user's sensitive features are extracted continuously and iteratively using the model to provide data for knowledge graph construction, which can solve the above-mentioned existing problems.

[0006] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0007] The method for dynamic measurement modeling of user privacy based on LSTM+CRF includes the following steps:

[0008] Step 1: Dynamic sensitive feature training text extraction;

[0009] S1: By designing an automatic crawler program based on the Scrapy framework;

[0010] S2: crawl user input text from mobile application services, and obtain privacy-related text annotation information from a trusted third-party library for comparison;

[0011] S3: Finally, store the private text information that the third-party library does not have into the CSV file;

[0012] Step 2: Training of optimized LSTM+CRF model;

[0013] S1: The text is processed through the embedding layer, LSTM layer, and conditional random field layer. The embedding layer is used to preprocess the text data so that the LSTM+CRF model can be trained using these texts. The LSTM layer is used to train the processed data to obtain the state of each word for each annotation, that is, the probability, and obtain the state score matrix. The CRF layer is used to iteratively train the state score matrix and find the annotation sentence with the minimum loss through the loss function, that is, the correctly annotated sentence.

[0014] S2: Segment the training text in CSV using the tokenizer algorithm and obtain the word embedding vector , through the semantic segmentation recognition algorithm, the huge amount of training text is segmented into training sentences according to punctuation, and the word vectors are used as the input of LSTM layer model training and input into the LSTM layer in batches, thereby improving the training efficiency of LSTM;

[0015] S3: After training, a variety of different LSTM layer state score matrices can be predicted , i.e. sensitive feature text, the probability matrix where each word is labeled with different parts of speech;

[0016] S4: LSTM layer state score matrix As the data support of CRF input, constraints are added to the training sentence annotations through the CRF layer, and the minimum loss function of a true path sequence is obtained through continuous iteration;

[0017] S5: Sensitive feature relationships are obtained through continuous optimization and iterative training of the LSTM layer and the CRF layer;

[0018] S6: Re-integrate the extracted user sensitive features;

[0019] S7: After removing redundancy from sensitive features, the refined sensitive features are stored in the user domain knowledge base;

[0020] S8: Each sensitive feature is labeled according to the storage time. Based on the user's sensitive features and their time in the knowledge base, the dynamic change trend of the user's sensitive features in a certain period of time can be predicted;

[0021] Step 3: Knowledge graph measurement and construction;

[0022] S1: The user sensitive features and sensitive feature relationships of the previous layer are input in the form of triples <sensitive feature 1, sensitive feature 2, sensitive feature relationship> according to the logical relationship, so that the machine can understand it more easily;

[0023] S2: According to the user's agreement on sensitive privacy protection rules, the user's sensitive features are measured for sensitivity, and the measured results are re-provided to the LSTM and CRF layers;

[0024] S3: During the training process, the measured data and the newly emerged knowledge logic in the network are continuously supplemented to further optimize the accuracy of the training model and scientifically reveal the dynamic change trend of user sensitive features.

[0025] As a preferred solution of the method for dynamic measurement modeling of user privacy based on LSTM+CRF described in the present invention, the sensitive information in the text crawled in S2 of step one will be stored in the form of text in the feature domain knowledge database as training text for the deep learning model.

[0026] As a preferred solution of the method for dynamic measurement modeling of user privacy based on LSTM+CRF described in the present invention, wherein: the real path sequence in S4 of the step 2 is that there must be a B label before each M label, and there must be a M or B label before the E label, etc. Fixed predicted labels, by labeling the correct path, that is, the loss function obtains the path with the minimum loss.

[0027] As a preferred solution of the method for dynamic measurement modeling of user privacy based on LSTM+CRF described in the present invention, the key steps of the CRF layer in S5 of step 2 can be calculated by the formula:

[0028] Wherein: the PRealpath represents the score of the true path, the Pi represents the scores of other paths, the LogLossFunction represents the loss function, and the TransitionScore represents the probability of state ti to ti+1.

[0029] As a preferred solution of the method for dynamic measurement modeling of user privacy based on LSTM+CRF described in the present invention, wherein: the fusion in step 2 S6 is to fuse sensitive features with a similarity of more than 80% through a similarity algorithm, only use one of the sensitive features, and delete redundant sensitive features.

[0030] As a preferred solution of the method for dynamic measurement modeling of user privacy based on LSTM+CRF described in the present invention, wherein: the triple in step three S1 is in the form of <entity 1, entity 2, relationship between entities>.

[0031] Compared with the existing technology: by using the Scrapy crawler to obtain user information on the Internet, and at the same time obtain manually labeled information about privacy from a trusted third party, and then train the model based on the LSTM+CRF model, a dynamic privacy measurement model is constructed, and the model is used to continuously iteratively extract the user's sensitive features, providing data for knowledge graph construction, thereby achieving the effect of optimizing the accuracy of the training model and scientifically revealing the dynamic change trend of user sensitive features. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The overall framework diagram provided for the present invention. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0034] This invention provides a method for dynamic measurement modeling of user privacy based on LSTM+CRF, please refer to Figure 1 , including the following steps:

[0035] Step 1: Dynamic sensitive feature training text extraction;

[0036] S1: By designing an automatic crawler program based on the Scrapy framework;

[0037] S2: crawl the text input by the user from the mobile application service, and obtain the privacy-related text annotation information from the trusted third-party library for comparison. The sensitive information in the crawled text will be stored in the feature domain knowledge database in the form of text as the training text for the deep learning model;

[0038] S3: Finally, store the private text information that the third-party library does not have into the CSV file;

[0039] Step 2: Training of optimized LSTM+CRF model;

[0040] Among them, the long short-term memory network, as an autoregressive model based on neural networks, occupies an indispensable position in deep learning and big data. It can well predict the model generation of characteristic behaviors such as time series data, text, and events;

[0041] S1: The text is processed through the embedding layer, LSTM layer, and conditional random field layer. The embedding layer is used to preprocess the text data so that the LSTM+CRF model can be trained using these texts. The LSTM layer is used to train the processed data to obtain the state of each word for each annotation, that is, the probability, and obtain the state score matrix. The CRF layer is used to iteratively train the state score matrix and find the annotation sentence with the minimum loss through the loss function, that is, the correctly annotated sentence.

[0042] In the embedding layer, the training text is preprocessed, the training text is annotated according to the requirements of sequence annotation, and the training text is more accurately annotated based on the privacy-related annotated text obtained from the trusted third-party library, so that the LSTM+CRF model can use these texts for training. The processed data is trained in the LSTM layer to obtain the state of each word for each annotation, that is, the probability, and obtain the state score matrix. The state score matrix is ​​iteratively trained in the CRF layer, and the annotated sentence with the smallest loss, that is, the correctly annotated sentence, is found through the loss function to achieve the persistence of user sensitive feature data. At the same time, considering the training efficiency of LSTM, this solution intends to split the obtained training text and annotate the data more accurately in the embedding layer, thereby improving the training accuracy of the LSTM+CRF model;

[0043] S2: Training text in CSV through word segmentation algorithm Perform word segmentation and obtain word embedding vector , through the semantic segmentation recognition algorithm, the huge amount of training text is segmented into training sentences according to punctuation, and the word vectors are used as the input of LSTM layer model training and input into the LSTM layer in batches, thereby improving the training efficiency of LSTM;

[0044] S3: After training, a variety of different LSTM layer state score matrices can be predicted , i.e. sensitive feature text, the probability matrix where each word is labeled with different parts of speech;

[0045] S4: LSTM layer state score matrix As the data support for CRF input, constraints are added to the training sentence annotations through the CRF layer, and the minimum loss function of a true path sequence is obtained through continuous iteration. The true path sequence is that there must be a B annotation before each M annotation, and there must be an M or B annotation before the E annotation. Fixed prediction annotations such as annotations are obtained by annotating the correct path, that is, the loss function obtains the path with the minimum loss, that is, the correct path with the minimum loss among all annotations, to ensure the legitimacy and accuracy of the prediction results of sensitive feature data;

[0046] S5: Through the continuous optimization and iterative training of the LSTM layer and the CRF layer, the sensitive feature relationship is obtained, that is, the relationship between multiple sensitive features. A single sensitive feature cannot reflect the user's privacy, and the relationship between multiple sensitive features can better reflect the connection between user privacy. The key steps of the CRF layer can be calculated by the formula:

[0047] PRealpath represents the score of the true path, Pi represents the score of other paths, LogLossFunction represents the loss function, and TransitionScore represents the probability of state ti to ti+1;

[0048] S6: Although the optimized bidirectional long short-term memory network model has advantages in sensitive feature extraction and feature relationship extraction, these results still contain a lot of redundant information. In order to improve the logic and hierarchy of sensitive features, it is necessary to fuse the extracted user sensitive features again. The fusion is to fuse sensitive features with a similarity of more than 80% through a similarity algorithm, only use one of the sensitive features, and delete redundant sensitive features, so as to achieve the purpose of improving the training quality of user sensitive feature data;

[0049] S7: After removing redundancy from sensitive features, the refined sensitive features are stored in the user domain knowledge base;

[0050] S8: Each sensitive feature is labeled according to the storage time. Based on the user's sensitive features and their time in the knowledge base, the dynamic change trend of the user's sensitive features in a certain period of time can be predicted;

[0051] Step 3: Knowledge graph measurement and construction;

[0052] The knowledge graph is a structured semantic knowledge base that aggregates a large amount of data knowledge by transitioning the data granularity from the file level to the data level, thus achieving rapid knowledge response and reasoning.

[0053] S1: The user sensitive features and sensitive feature relationships of the previous layer are input in the form of triples <sensitive feature 1, sensitive feature 2, sensitive feature relationship> according to the logical relationship, so that the machine can understand it more easily. The triples are in the form of <entity 1, entity 2, relationship between entities>;

[0054] S2: According to the user's agreement on sensitive privacy protection rules, the user's sensitive features are measured for sensitivity, and the measured results are re-provided to the LSTM and CRF layers. The measured results are more accurate and can be used for training to improve the accuracy of the model in extracting user sensitive attributes.

[0055] S3: During the training process, the measured data and the newly emerged knowledge logic in the network are continuously supplemented to further optimize the accuracy of the training model and scientifically reveal the dynamic change trend of user sensitive features.

[0056] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for dynamic measurement modeling of user privacy based on LSTM+CRF, characterized by: The steps include: Step 1: Dynamic sensitive feature training text extraction; S1: By designing an automatic crawler program based on the Scrapy framework; S2: crawl user input text from mobile application services, and obtain privacy-related text annotation information from a trusted third-party library for comparison; S3: Finally, store the private text information that the third-party library does not have into the CSV file; Step 2: Training of optimized LSTM+CRF model; S1: The text is processed through the embedding layer, LSTM layer, and conditional random field layer. The embedding layer is used to preprocess the text data so that the LSTM+CRF model can be trained using these texts. The LSTM layer is used to train the processed data to obtain the state of each word for each annotation, that is, the probability, and obtain the state score matrix. The CRF layer is used to iteratively train the state score matrix and find the annotation sentence with the minimum loss through the loss function, that is, the correctly annotated sentence. S2: Training text in CSV through word segmentation algorithm Perform word segmentation and obtain word embedding vector , through the semantic segmentation recognition algorithm, the huge amount of training text is segmented into training sentences according to punctuation, and the word vectors are used as the input of LSTM layer model training and input into the LSTM layer in batches, thereby improving the training efficiency of LSTM; S3: After training, a variety of different LSTM layer state score matrices can be predicted , i.e. sensitive feature text, each word is labeled with a probability matrix of different parts of speech; S4: LSTM layer state score matrix As the data support of CRF input, constraints are added to the training sentence annotations through the CRF layer, and the minimum loss function of a true path sequence is obtained through continuous iteration; S5: Sensitive feature relationships are obtained through continuous optimization and iterative training of the LSTM layer and the CRF layer; S6: Re-integrate the extracted user sensitive features; S7: After removing redundancy from sensitive features, the refined sensitive features are stored in the user domain knowledge base; S8: Each sensitive feature is labeled according to the storage time. Based on the user's sensitive features and their time in the knowledge base, the dynamic change trend of the user's sensitive features in a certain period of time can be predicted; Step 3: Knowledge graph measurement and construction; S1: The user sensitive features and sensitive feature relationships of the previous layer are input in the form of triples <sensitive feature 1, sensitive feature 2, sensitive feature relationship> according to the logical relationship, so that the machine can understand it more easily; S2: According to the user's agreement on sensitive privacy protection rules, the user's sensitive features are measured for sensitivity, and the measured results are re-provided to the LSTM and CRF layers; S3: During the training process, the measured data and the newly emerged knowledge logic in the network are continuously supplemented to further optimize the accuracy of the training model and scientifically reveal the dynamic change trend of user sensitive features.

2. The method for dynamic measurement modeling of user privacy based on LSTM+CRF according to claim 1 is characterized in that: The sensitive information in the text crawled in S2 of step 1 will be stored in the feature domain knowledge database in the form of text as training text for the deep learning model.

3. The method for dynamic measurement modeling of user privacy based on LSTM+CRF according to claim 1 is characterized in that: The real path sequence in S4 of step 2 is that there must be a B label before each M label, and there must be an M or B label before the E label, etc. The correct path is labeled, that is, the loss function obtains the path with the minimum loss.

4. The method for dynamic measurement modeling of user privacy based on LSTM+CRF according to claim 1 is characterized in that: The key steps of the CRF layer in S5 of step 2 can be calculated by the formula: Wherein the PRealpath represents the score of the true path, the Pi represents the scores of other paths, the LogLossFunction represents the loss function, and the TransitionScore represents the probability of state ti to ti+1.

5. The method for dynamic measurement modeling of user privacy based on LSTM+CRF according to claim 1 is characterized in that: In the step 2 S6, the fusion is to fuse the sensitive features with a similarity of more than 80% through a similarity algorithm, use only one of the sensitive features, and delete redundant sensitive features.

6. The method for dynamic measurement modeling of user privacy based on LSTM+CRF according to claim 1 is characterized in that: The triplet in step three S1 is in the form of <entity 1, entity 2, relationship between entities>.

Citation Information

Patent Citations

  • Sensitive information monitoring method based on event search

    CN102693314A

  • Entity identification method, device, device and storage medium

    CN109299458A

  • A method for establishing a detection model of covert sensitive text for network social media

    CN109543084A

  • Query and visualization system construction method based on minority culture knowledge graph

    CN111143574A

  • Building domain knowledge graph construction method based on neural network self-adaptive optimization parameter adjustment

    CN111813950A