System for protecting sensitive information of large model
Through user identity registration, access control, sensitive information database and behavior monitoring, combined with natural language processing and machine learning, the problem of sensitive information identification and protection when large models generate content is solved, and the accurate identification and security of sensitive information is achieved. Control enhances the security and data protection capabilities of the system.
Patent Information
- Application Number
- CN202411040781.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-19
AI Technical Summary
Large models cannot distinguish sensitive information when generating content, the existing technology cannot effectively conduct user permission checks and sensitive information screening, and traditional NLP methods have errors in semantic understanding, resulting in a high risk of leakage of sensitive information.
Comprehensive measures of user identity registration and authorization management, access control policy formulation, sensitive information database construction, sensitive information inspection of input and generated content, user behavior monitoring and risk assessment, logging and security auditing, through natural language processing and machine learning The algorithm performs sensitive information identification and intercepting.
It effectively protects sensitive information in large-scale applications, ensures that users only access data within the scope of authorization, monitors and evaluates user behavior in real time, improves system security and data integrity, and prevents sensitive information leakage.
Smart Images

Figure CN118981789A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to a system for protecting sensitive information of a large model. Background Art
[0002] With the widespread application of artificial intelligence technologies such as big models, their ability to generate content has provided great convenience to users. However, due to the technical characteristics of big models that generate content based on training data, they cannot distinguish the sensitivity of the generated content. Therefore, in units where sensitive information is concentrated, how to achieve security control of its application while using big models, avoid the leakage of sensitive information, and protect the security of information and data has become an urgent problem to be solved.
[0003] The existing technology has two defects:
[0004] First, after the big model generated content, there was no check on sensitive words, no check on the user permissions and usage rules of the big model, no authorization on the scope of use, and no interception of possible sensitive information attempts.
[0005] Second, the existing sensitive word screening uses traditional NLP, which performs word comparison based on the generated text content. It cannot judge whether the same word is a sensitive word in the context based on the understanding of the contextual semantics, and the error in sensitive word screening is large. Summary of the invention
[0006] The purpose of the invention is to provide a system for protecting sensitive information of large models, which solves the problems in the background technology.
[0007] The present invention is implemented as follows: a system for protecting sensitive information of a large model, the system for protecting sensitive information of a large model comprising the following steps:
[0008] S1. User identity registration and authorization management;
[0009] S2. Development of access control strategy;
[0010] S3, construction of sensitive information database;
[0011] S4, sensitive information check of input and generated content;
[0012] S5. User behavior monitoring and risk assessment;
[0013] S6. Logging and security auditing.
[0014] A further technical solution of the present invention is: the user identity registration and authorization management is: the customer provides detailed identity information, including but not limited to name, level, business scope, and department to which he belongs. The system assigns corresponding access rights and roles to the user based on the information provided by the user; the user's access rights can be defined based on multiple dimensions such as role, department, level, etc., to ensure that different users can only access information within their authorized scope.
[0015] A further technical solution of the present invention is that the access control policy is formulated by formulating rules for large model access based on the information provided by the user during registration; the rules may include the type of data, time range, geographic location, etc. that the user can access. The access control policy can be dynamically configured and managed through a rule policy engine to adapt to different business needs and scenario changes.
[0016] A further technical solution of the present invention is: the construction of the sensitive information database is: establishing a sensitive information database, including the level, classification and directory of sensitive information. Sensitive information may include personal privacy, commercial secrets, and state secrets; the sensitive information database needs to be regularly updated and maintained to ensure its accuracy and integrity. The sensitive information database can establish a corresponding relationship with the access control policy to form a sensitive information access policy.
[0017] A further technical solution of the present invention is: to perform a sensitive word check on the information input by the user to check whether it contains keywords or phrases in the sensitive information library; and to perform a similar check on the content generated by the large model to ensure that the generated content does not contain sensitive information.
[0018] A further technical solution of the present invention is: the sensitive information check of the input and generated content is: content extraction and semantic understanding and vector comparison; if sensitive information is found, the system will intercept it according to the sensitive information access policy or prompt the user to modify it.
[0019] A further technical solution of the present invention is: the extraction of the content is: extracting sensitive words and phrases through natural language processing technology and machine learning algorithms, confirming the text content of sensitive information, and calculating the semantic vectors of sensitive words and phrases.
[0020] A further technical solution of the present invention is: the semantic understanding and vector comparison are: understanding the context semantics of sensitive words through a large model, and performing vector comparison on the context vector and the sensitive information library to determine the vector value similarity. If the similarity exceeds a threshold range, it is considered to be sensitive information.
[0021] A further technical solution of the present invention is: user behavior monitoring and risk assessment are: real-time monitoring and recording of user behavior using large models; including the type of data accessed, time, location, etc.
[0022] Risk assessment is performed based on user behavior records and combined with the rule policy engine. Assessment indicators may include access frequency, sensitivity of access data, abnormal operations, etc. If abnormal or high-risk behavior is found, the system will issue a risk warning and may take measures such as terminating access authorization.
[0023] A further technical solution of the present invention is: logging and security auditing are: logging and security auditing are: logging is to make detailed log records of various behaviors, input and output results of the large model usage process; logging should contain sufficient information for subsequent security audits and problem analysis; security auditing is: logs can be reviewed and analyzed regularly to discover potential security issues and vulnerabilities; audit results can be used to optimize access control strategies, improve sensitive information checking algorithms, etc., to improve the security and reliability of the system.
[0024] The beneficial effects of the present invention are as follows: it effectively improves the sensitive information protection capability of large model applications, provides a reliable technical guarantee for the safe use of large models, prevents the leakage of sensitive information, and enhances the security of the system and data.
[0025] Its advantages mainly include the following aspects:
[0026] Sensitive information protection: By implementing the method, system and device of this patent, effective protection of sensitive information in the application process of large models can be achieved. The introduction of sensitive information library and sensitive information inspection module enables the system to accurately identify and process sensitive information to avoid its leakage or improper use.
[0027] Access control strategy: The formulation and implementation of detailed access control strategies ensure that different users can only access data within their authorized scope. This multi-dimensional access control based on user identity, business scope, level, etc. greatly improves the security of the system.
[0028] User behavior monitoring and risk assessment: Real-time monitoring and risk assessment of user behavior can promptly detect abnormal or high-risk behavior and take appropriate measures to intervene, further improving the security of the system.
[0029] Logging and security auditing: The system records all operation logs such as user behavior, input content, generated content, etc. in detail, and conducts regular or irregular review and analysis through the security audit function. This helps to discover potential security issues and vulnerabilities and repair them in a timely manner, thereby enhancing data security. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a flow chart of a system for protecting sensitive information of a large model provided by the present invention;
[0031] Figure 2 It is a flow chart of the implementation steps of a system for protecting sensitive information of a large model provided by the present invention. DETAILED DESCRIPTION
[0032] Embodiment 1: Figure 1-Figure 2 A system for protecting sensitive information of a large model is shown, and the system for protecting sensitive information of a large model includes the following steps: The system for protecting sensitive information of a large model includes the following steps:
[0033] S1. User identity registration and authorization management;
[0034] S2. Development of access control strategy;
[0035] S3, construction of sensitive information database;
[0036] S4, sensitive information check of input and generated content;
[0037] S5. User behavior monitoring and risk assessment;
[0038] S6. Logging and security auditing.
[0039] 1. User identity registration and authorization management:
[0040] Users register through the system interface and fill in detailed information including name, level, business scope, department, etc. The system automatically assigns initial access rights and roles to users based on the information filled in by users, ensuring that users can only operate within their authorized scope.
[0041] 2. Development of access control strategy:
[0042] System administrators use the backend management system to develop detailed access control policies based on business needs. Policies can include: the correspondence between user levels and access rights, the correspondence between business scope and data type, the correspondence between time range and access rights, and the correspondence between geographic location and access rights. Access control policies are dynamically managed and adjusted through the rule policy engine.
[0043] 3. Construction of sensitive information database:
[0044] System administrators build sensitive information libraries, including:
[0045] Sensitivity level: Level 1 (highly sensitive), Level 2 (sensitive), Level 3 (normal), etc.
[0046] Sensitive categories: personal privacy (such as ID number, mobile phone number, etc.), business secrets (such as contract content, financial data, etc.), state secrets (such as military and diplomatic information, etc.).
[0047] Sensitive word semantics: the semantic vector of sensitive words in the natural language context.
[0048] The sensitive information library supports regular updates and manual addition of new sensitive information.
[0049] 4. Check sensitive information of input and generated content
[0050] The information input by the user and the content generated by the large model are checked for sensitive words, compared with the sensitive information database, and the sensitivity of the same word under different semantics is identified through semantic vectors. If sensitive information is found, the system will take corresponding measures according to the access control policy and the level of sensitive information, such as interception, modification prompts or automatic desensitization.
[0051] 5. User behavior monitoring and risk assessment
[0052] The system monitors and records user access behaviors in real time, including access time, data type, operation content, etc. Through behavioral analysis algorithms, combined with access control policies and sensitive information libraries, risk assessment of user behaviors is performed. If abnormal or high-risk behaviors are found, the system will immediately issue risk warnings, such as sending alert emails, pop-up prompt boxes, etc., and may take measures such as terminating access authorization.
[0053] 6. Logging and security auditing
[0054] The system records all operation logs including user behavior, input content, generated content, etc. in detail.
[0055] Log records include key information such as timestamp, user ID, operation type, and operation content.
[0056] The system provides a security audit function, and administrators can review and analyze logs regularly or irregularly to identify potential security issues and vulnerabilities.
[0057] Audit results can be used to optimize access control policies, improve sensitive information checking algorithms, etc., to improve the security and reliability of the system.
[0058] The present invention provides a system for protecting sensitive information of large models. Through measures such as authorization management, access control, sensitive information inspection, behavior monitoring, and security auditing for users, the system can achieve security management and control of large model applications, ensure system security and data security, and enable large model applications to meet the technical specifications for security protection.
[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A system for protecting sensitive information of a large model, characterized by: The system for protecting sensitive information of a large model includes the following steps: S1. User identity registration and authorization management; S2. Development of access control strategy; S3, construction of sensitive information database; S4, sensitive information check of input and generated content; S5. User behavior monitoring and risk assessment; S6. Logging and security auditing.
2. A system for protecting sensitive information of a large model according to claim 1, characterized in that: The user identity registration and authorization management is as follows: the customer provides detailed identity information, including but not limited to name, level, business scope, and department to which he belongs. The system assigns corresponding access rights and roles to the user based on the information provided by the user.
3. A system for protecting sensitive information of a large model according to claim 1, characterized in that: The access control strategy is formulated as follows: based on the information provided by the user during registration, rules for accessing the large model are formulated.
4. A system for protecting sensitive information of a large model according to claim 1, characterized in that: The construction of the sensitive information database is as follows: establishing a sensitive information database, including the level, classification and directory of sensitive information; sensitive information may include personal privacy, commercial secrets, and state secrets.
5. A system for protecting sensitive information of a large model according to claim 1, characterized in that: Perform a sensitive word check on the information entered by the user to check whether it contains keywords or phrases in the sensitive information database.
6. A system for protecting sensitive information of a large model according to claim 5, characterized in that: The sensitive information check of the input and generated content includes: content extraction and semantic understanding and vector comparison.
7. A system for protecting sensitive information of a large model according to claim 6, characterized in that: The extraction of the content is: extracting sensitive words and phrases through natural language processing technology and machine learning algorithms, confirming the text content of sensitive information, and calculating the semantic vectors of sensitive words and phrases.
8. A system for protecting sensitive information of a large model according to claim 6, characterized in that: The semantic understanding and vector comparison are as follows: the context semantics of sensitive words are understood through a large model, and the context vector is compared with the sensitive information library to determine the vector value similarity. If the similarity exceeds a threshold range, it is considered to be sensitive information.
9. A system for protecting sensitive information of a large model according to claim 8, characterized in that: User behavior monitoring and risk assessment include real-time monitoring and recording of user behavior using large models.
10. A system for protecting sensitive information of a large model according to claim 9, characterized in that: Logging and security auditing are: logging is to record in detail the various behaviors, input and output results of the large model usage process; security auditing is to review and analyze the logs regularly to discover potential security issues and vulnerabilities.
Citation Information
Cited By
Security monitoring method and system based on large model embedded in smart screen
CN120296749B
Large model privacy protection method and device
CN121256859A
A method and apparatus for protecting privacy in large models
CN121256859B
