Large model-based recognition and answer rejection method and system
Through the rejection method and system based on large-model, preprocessing and large-model embedding vector technology, the problem that traditional network security is difficult to deal with complex attacks is solved, intelligent interception and protection are achieved, and the security of the network environment is improved.
Patent Information
- Application Number
- CN202411196566.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional cybersecurity protection methods are difficult to deal with complex and changeable cyber attacks and threats, especially in the application of large-scale data processing and deep learning models, and it is difficult to effectively identify and intercept malicious requests and protect user data from attacks.
Using a rejection method and system based on big model, the problem text submitted by the user is preprocessed, and the pre-trained big model is used to convert it into an embedded vector, and an index library is built to calculate the distance value from illegal content. Trigger rejection or return answer based on the set distance threshold to achieve intelligent interception and protection.
The harmony and security level of the network environment have been improved, and various types of network threats are effectively responded to through intelligent interception functions, and the identification and interception capabilities of illegal requests have been improved.
Smart Images

Figure CN119090692A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network security technology, and in particular to a large model-based method and system for rejecting recognition and answering. Background Art
[0002] With the rapid development of artificial intelligence technology, deep learning models are increasingly used in various fields, such as natural language processing (NLP) and computer vision (CV). Especially driven by large-scale models, such as the Transformer model and its GPT, BERT, etc., these models have demonstrated unprecedented performance improvements through huge parameter scales and massive training data, and can handle more complex and diverse tasks. They are now widely used in various fields. However, with the widespread application of large models, network security issues are becoming increasingly prominent. Traditional network security protection methods are often difficult to cope with complex and changing network attacks and threats. In particular, for large-scale data processing and the application of deep learning models, the demand for preventing illegal issues is growing. While processing massive amounts of data, large models also face the risk of being maliciously exploited. In the field of network security, identifying and intercepting malicious requests and protecting user data from attacks is a vital task. Although traditional network security protection methods, such as firewalls and intrusion detection systems, can protect system security to a certain extent, their effectiveness is often limited in the face of complex and changing network attacks.
[0003] In recent years, the technology of denying recognition and refusing to answer based on big data and deep learning models has gradually become a research hotspot in the field of network security. Denial of recognition and refusing to answer is a security strategy based on AI and big data technology, which is used to identify and refuse to answer questions that may cause security issues, privacy leaks or ethical disputes. Summary of the invention
[0004] In view of the needs and shortcomings of current technological development, the present invention provides a method and system for rejecting recognition and rejecting answers based on a large model, aiming to improve the harmony and security level of the network environment and effectively deal with various network threats through intelligent interception functions.
[0005] In the first aspect, the present invention provides a method for rejecting recognition and rejecting answering based on a large model, and the technical solution adopted to solve the above technical problems is as follows:
[0006] A method for rejecting recognition and rejecting answering based on a large model comprises the following steps:
[0007] S1. Preprocess the question text submitted by the user;
[0008] S2. Select and load the pre-trained large model, which converts the pre-processed question text into an embedding vector;
[0009] S3, define illegal content and build an index library;
[0010] S4. Calculate the distance between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result.
[0011] Optionally, the step S3 involved specifically includes:
[0012] S3.1. Define illegal content based on business needs and relevant laws and regulations;
[0013] S3.2. Organize and store the identified illegal content into a spreadsheet named rejections.xlsx;
[0014] S3.3, using the pre-trained large model selected in step S2 to encode each illegal content in the spreadsheet rejections.xlsx, convert it into a vector form, and generate an embedding.pkl file, which encapsulates the semantic features of all illegal content;
[0015] S3.4. Import the embedding.pkl file into ChromaDB and build an index for the imported vector in ChromaDB.
[0016] Optionally, the step S4 involved specifically includes:
[0017] S4.1. Based on the results of historical data testing, set the distance threshold;
[0018] S4.2. For the embedding vector corresponding to the preprocessed problem text, execute the nearest neighbor search algorithm in ChromaDB, calculate the distance between the current embedding vector and each vector-based illegal content in ChromaDB, and sort the distance values in ascending order;
[0019] S4.3. When the calculated distance value is less than the set distance threshold, the rejection condition is triggered, and the question text is automatically judged to have illegal risks, and then marking or interception measures are taken, and prompt information is fed back to the corresponding user;
[0020] S4.4. When the calculated distance value is not less than the set distance threshold, the answer corresponding to the question text is returned to the user through an online search or a pre-built database.
[0021] Preferably, the question text submitted by the user, as well as the marked or blocked content, is manually reviewed, and the set distance threshold is adjusted according to the false alarm rate and missed alarm rate of the question text:
[0022] If the false positive rate of the audit results is high, increase the distance threshold;
[0023] If the audit result has a high false negative rate, lower the distance threshold.
[0024] Optionally, executing step S1, performing preprocessing operations such as removing punctuation marks, stop words, text normalization, stemming or lemmatization on the question text submitted by the user;
[0025] Execute step S2, the large model is a BGE model.
[0026] In a second aspect, the present invention provides a large model-based rejection and rejection system, and the technical solution adopted to solve the above technical problems is as follows:
[0027] A large model-based rejection and rejection system, comprising:
[0028] A preprocessing module is used to preprocess the question text submitted by the user;
[0029] Select the loading module to select and load the pre-trained large model;
[0030] A large model for converting preprocessed question text into embedding vectors;
[0031] Define building blocks to define illegal content and build an index library;
[0032] The calculation return module is used to calculate the distance value between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result.
[0033] Optionally, the definition building modules involved specifically include:
[0034] Definition unit, used to define illegal content according to business needs and relevant laws and regulations;
[0035] A storage unit for collating and storing the identified illegal contents into a spreadsheet rejections.xlsx;
[0036] The encoding generation unit is used to encode each illegal content in the electronic form rejections.xlsx using the loaded large model, convert it into a vector form, and generate an embedding.pkl file. The embedding.pkl file encapsulates the semantic features of all illegal contents;
[0037] Import building unit, used to import the embedding.pkl file into ChromaDB and build indexes for the imported vectors in ChromaDB.
[0038] Optionally, the calculation return module involved specifically includes:
[0039] A threshold setting unit, used for setting a distance threshold based on the result of the historical data test;
[0040] An algorithm execution unit, used to execute a nearest neighbor search algorithm in ChromaDB, and calculate the distance between the embedding vector corresponding to the preprocessed problem text and each vector-based illegal content in ChromaDB;
[0041] A sorting unit, used to sort the distance values in ascending order;
[0042] A judgment unit, used to calculate whether the obtained distance value is greater than a set distance threshold;
[0043] A trigger prompt unit is used to trigger a rejection condition when the calculated distance value is less than a set distance threshold, automatically determine that the question text has illegal risks, then take marking or interception measures, and feedback prompt information to the corresponding user;
[0044] The answer returning module is used to return the answer corresponding to the question text to the user through online search or a pre-built database when the calculated distance value is not less than a set distance threshold.
[0045] Preferably, the question text submitted by the user, as well as the marked or blocked content, is manually reviewed, and the set distance threshold is adjusted according to the false alarm rate and missed alarm rate of the question text:
[0046] If the false alarm rate of the audit result is high, the distance threshold is increased through the threshold setting unit;
[0047] If the false negative rate of the audit result is high, the distance threshold is lowered through the threshold setting unit.
[0048] Optionally, the preprocessing module performs preprocessing operations such as removing punctuation marks, stop words, text normalization, stemming or lemmatization on the question text submitted by the user;
[0049] The involved selection load module selects and loads the BGE model.
[0050] Compared with the prior art, the method and system for rejecting recognition and answering based on a large model of the present invention has the following beneficial effects:
[0051] The present invention can improve the harmony and security level of the network environment and effectively deal with various network threats through the intelligent interception function. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Attached Figure 1 is a flow chart of a method according to Embodiment 1 of the present invention;
[0053] Attached Figure 2 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.
[0055] Embodiment 1:
[0056] Combined with Figure 1 This embodiment proposes a method for rejecting recognition and rejecting answering based on a large model, which includes the following steps:
[0057] S1. Preprocess the question text submitted by the user by removing punctuation marks, stop words, normalizing the text, extracting stems or restoring the word forms.
[0058] S2. Select and load a pre-trained large model. The large model selected in this embodiment is the BGE model. The BGE model (BaseEmbedding Generator) is a deep embedding generator for text and images. It is a series of general semantic vector models open sourced by the Beijing Academy of Artificial Intelligence (BAAI). It aims to improve the performance of tasks such as search, recommendation, and data mining by optimizing semantic vector representation.
[0059] The BGE model converts the preprocessed question text into an embedding vector.
[0060] S3: Define illegal content and build an index library. The specific operations include:
[0061] S3.1. Define illegal content based on business needs and relevant laws and regulations;
[0062] S3.2. Organize and store the identified illegal content into a spreadsheet named rejections.xlsx;
[0063] S3.3, using the BGE model selected in step S2 to encode each illegal content in the spreadsheet rejections.xlsx, converting it into a vector form, and generating an embedding.pkl file, which encapsulates the semantic features of all illegal content;
[0064] S3.4. Import the embedding.pkl file into ChromaDB and build an index for the imported vector in ChromaDB.
[0065] S4. Calculate the distance between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result. The specific operations include:
[0066] S4.1. Based on the results of historical data testing, set the distance threshold;
[0067] S4.2. For the embedding vector corresponding to the preprocessed problem text, execute the nearest neighbor search algorithm in ChromaDB, calculate the distance between the current embedding vector and each vector-based illegal content in ChromaDB, and sort the distance values in ascending order;
[0068] S4.3. When the calculated distance value is less than the set distance threshold, the rejection condition is triggered, and the question text is automatically judged to have illegal risks. Then, marking or interception measures are taken, and feedback prompt information is given to the corresponding user, such as: "There is a security problem with this question, please ask again." Such instant feedback helps users understand and correct their content, and also effectively blocks the spread of potential illegal information;
[0069] S4.4. When the calculated distance value is not less than the set distance threshold, the answer corresponding to the question text is returned to the user through an online search or a pre-built database.
[0070] It should be added that the question texts submitted by users, as well as the marked or blocked content, are manually reviewed, and the set distance threshold is adjusted according to the false positive rate and false negative rate of the question texts:
[0071] If the false positive rate of the audit results is high, increase the distance threshold;
[0072] If the audit result has a high false negative rate, lower the distance threshold.
[0073] Embodiment 2:
[0074] Combined with Figure 2 This embodiment proposes a large model-based rejection and rejection system, which includes:
[0075] A preprocessing module is used to preprocess the question text submitted by the user by removing punctuation marks, stop words, normalizing the text, extracting stems or restoring the word form;
[0076] Select the loading module to select and load the pre-trained large model;
[0077] A large model for converting preprocessed question text into embedding vectors;
[0078] Define building blocks to define illegal content and build an index library;
[0079] The calculation return module is used to calculate the distance value between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result.
[0080] In this embodiment, the selected large model is the BGE model. The BGE model (Base Embedding Generator) is a deep embedding generator for text and pictures. It is a series of general semantic vector models open sourced by the Beijing Academy of Artificial Intelligence (BAAI). It aims to improve the performance of tasks such as search, recommendation, and data mining by optimizing semantic vector representation.
[0081] In this embodiment, the definition building modules involved specifically include:
[0082] Definition unit, used to define illegal content according to business needs and relevant laws and regulations;
[0083] A storage unit for collating and storing the identified illegal contents into a spreadsheet rejections.xlsx;
[0084] The encoding generation unit is used to encode each illegal content in the electronic form rejections.xlsx using the loaded large model, convert it into a vector form, and generate an embedding.pkl file. The embedding.pkl file encapsulates the semantic features of all illegal contents;
[0085] Import building unit, used to import the embedding.pkl file into ChromaDB and build indexes for the imported vectors in ChromaDB.
[0086] In this embodiment, the calculation return module involved specifically includes:
[0087] A threshold setting unit, used for setting a distance threshold based on the result of the historical data test;
[0088] An algorithm execution unit, used to execute a nearest neighbor search algorithm in ChromaDB, and calculate the distance between the embedding vector corresponding to the preprocessed problem text and each vector-based illegal content in ChromaDB;
[0089] A sorting unit, used to sort the distance values in ascending order;
[0090] A judgment unit, used to calculate whether the obtained distance value is greater than a set distance threshold;
[0091] A trigger prompt unit is used to trigger a rejection condition when the calculated distance value is less than a set distance threshold, automatically determine that the question text has illegal risks, then take marking or interception measures, and feedback prompt information to the corresponding user;
[0092] The answer returning module is used to return the answer corresponding to the question text to the user through online search or a pre-built database when the calculated distance value is not less than a set distance threshold.
[0093] In this embodiment, the distance threshold set by the threshold setting unit is initially a static value, but in subsequent use, the distance threshold is not fixed. After the distance threshold is initially set, the question text submitted by the user and the marked or intercepted content are manually reviewed, and the set initial distance threshold is adjusted according to the false alarm rate and missed alarm rate of the question text:
[0094] If the false alarm rate of the audit result is high, the distance threshold is increased through the threshold setting unit;
[0095] If the false negative rate of the audit result is high, the distance threshold is lowered through the threshold setting unit.
[0096] Continuous adjustment and optimization of this distance threshold can better achieve problem classification to mark or intercept problematic texts that pose illegal risks.
[0097] In summary, the large-model-based method and system for refusing to recognize and answer messages of the present invention can improve the harmony and security level of the network environment, and effectively respond to various network threats through intelligent interception functions.
[0098] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.
Claims
1. A method for rejecting recognition and rejecting answering based on a large model, characterized in that: The steps include: S1. Preprocess the question text submitted by the user; S2. Select and load the pre-trained large model, which converts the pre-processed question text into an embedding vector; S3, define illegal content and build an index library; S4. Calculate the distance between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result.
2. A method for rejecting recognition and rejecting answering based on a large model according to claim 1, characterized in that: The step S3 specifically includes: S3.
1. Define illegal content based on business needs and relevant laws and regulations; S3.
2. Organize and store the identified illegal content into a spreadsheet named rejections.xlsx; S3.3, using the pre-trained large model selected in step S2 to encode each illegal content in the spreadsheet rejections.xlsx, convert it into a vector form, and generate an embedding.pkl file, which encapsulates the semantic features of all illegal content; S3.
4. Import the embedding.pkl file into ChromaDB and build an index for the imported vector in ChromaDB.
3. The method for rejecting recognition and rejecting answering based on a large model according to claim 1, characterized in that: The step S4 specifically includes: S4.
1. Based on the results of historical data testing, set the distance threshold; S4.
2. For the embedding vector corresponding to the preprocessed problem text, execute the nearest neighbor search algorithm in ChromaDB, calculate the distance between the current embedding vector and each vector-based illegal content in ChromaDB, and sort the distance values in ascending order; S4.
3. When the calculated distance value is less than the set distance threshold, the rejection condition is triggered, and the question text is automatically judged to have illegal risks, and then marking or interception measures are taken, and prompt information is fed back to the corresponding user; S4.
4. When the calculated distance value is not less than the set distance threshold, the answer corresponding to the question text is returned to the user through an online search or a pre-built database.
4. The method for rejecting recognition and rejecting answering based on a large model according to claim 3, characterized in that: Manually review the question texts submitted by users, as well as the marked or blocked content, and adjust the set distance threshold based on the false positive rate and false negative rate of the question text: If the false positive rate of the audit results is high, increase the distance threshold; If the audit result has a high false negative rate, lower the distance threshold.
5. The method for rejecting recognition and rejecting answering based on a large model according to claim 1, characterized in that: Execute step S1 to perform preprocessing operations such as removing punctuation marks, stop words, text normalization, stemming or lemmatization on the question text submitted by the user; Execute step S2, the large model is a BGE model.
6. A large model-based rejection and rejection system, characterized in that: It includes: A preprocessing module is used to preprocess the question text submitted by the user; Select the loading module to select and load the pre-trained large model; A large model for converting preprocessed question text into embedding vectors; Define building blocks to define illegal content and build an index library; The calculation return module is used to calculate the distance value between the question text submitted by the user and the illegal content in the index library, and return information to the user based on the calculation result.
7. A large model-based rejection and rejection system according to claim 6, characterized in that: The definition building module specifically includes: Definition unit, used to define illegal content according to business needs and relevant laws and regulations; A storage unit for collating and storing the identified illegal contents into a spreadsheet rejections.xlsx; The encoding generation unit is used to encode each illegal content in the electronic form rejections.xlsx using the loaded large model, convert it into a vector form, and generate an embedding.pkl file. The embedding.pkl file encapsulates the semantic features of all illegal contents; Import building unit, used to import the embedding.pkl file into ChromaDB and build indexes for the imported vectors in ChromaDB.
8. The large model-based rejection and rejection system according to claim 6, characterized in that: The calculation return module specifically includes: A threshold setting unit, used for setting a distance threshold based on the result of the historical data test; An algorithm execution unit, used to execute a nearest neighbor search algorithm in ChromaDB, and calculate the distance between the embedding vector corresponding to the preprocessed problem text and each vector-based illegal content in ChromaDB; A sorting unit, used to sort the distance values in ascending order; A judgment unit, used to calculate whether the obtained distance value is greater than a set distance threshold; A trigger prompt unit is used to trigger a rejection condition when the calculated distance value is less than a set distance threshold, automatically determine that the question text has illegal risks, then take marking or interception measures, and feedback prompt information to the corresponding user; The answer returning module is used to return the answer corresponding to the question text to the user through online search or a pre-built database when the calculated distance value is not less than a set distance threshold.
9. A large model-based rejection and rejection system according to claim 8, characterized in that: Manually review the question texts submitted by users, as well as the marked or blocked content, and adjust the set distance threshold based on the false positive rate and false negative rate of the question text: If the false alarm rate of the audit result is high, the distance threshold is increased through the threshold setting unit; If the false negative rate of the audit result is high, the distance threshold is lowered through the threshold setting unit.
10. The large model-based rejection and rejection system according to claim 6, characterized in that: The preprocessing module performs preprocessing operations such as removing punctuation marks, stop words, text normalization, stem extraction or morphological restoration on the question text submitted by the user; The selection and loading module selects and loads the BGE model.
Citation Information
Cited By
Man-machine interaction method and system based on AI large model
CN119646167A
