Intelligent question and answer data processing system based on AI

Abstract

The invention relates to the field of data processing, in particular to an AI-based intelligent question and answer data processing system, which comprises the following steps of: recording a plurality of sensitive information databases for storing sensitive information and the permission range of each user side for the sensitive information; obtaining text data output by a user side, and determining the local association degree of the keyword relative to the adjacent keyword so as to determine the local association difference of the text data; determining explanation keywords, and calculating an association degree mean value of the explanation keywords, the corresponding derivative words and the remaining keywords with the sensitive information outside the permission range of the user side; determining a semantic sensitive tendency characterization value based on the local association difference and the association degree mean value, and dividing semantic sensitive tendency categories of the user side; adaptively processing the text data of the user side based on the semantic sensitive tendency category of the user side; and receiving feedback information of the language model, and transmitting the feedback information to the corresponding user side, so that the reliability of reply of the language model can be improved, and privacy data leakage is avoided.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an AI-based intelligent question-answering data processing system. Background Art

[0002] AI-based intelligent question and answer is a fusion of multiple technologies to understand user questions, retrieve, generate answers and feed them back to the user. Specifically, in the analysis of question content, natural language processing technology is used to identify and understand the semantics of the question content, and then retrieve and generate reply feedback data for the question content. At present, some companies have customized language models to answer employee questions. The data used in the training of such language models is relatively comprehensive, and the language model can answer various questions. However, when the large model analyzes semantics, the input data will be transmitted and transformed through multiple layers of networks. Each layer will perform complex nonlinear mapping on the data and finally obtain the output result. Therefore, complex reasoning may cause the large model to output sensitive content. Furthermore, there may be a situation where the language model is induced to output sensitive content through the distortion of question and answer sentences. In particular, different employees have different permissions. When providing intelligent question and answer services, it is necessary to consider the issues of true semantic understanding and data privacy protection.

[0003] Chinese patent application publication number: CN116910223A, discloses an intelligent question-answering data processing system based on a pre-trained model, which includes a data collection module, a pre-trained model module, a fine-tuning module, an inference module, a post-processing module and a user state recognition module. By training the pre-trained model in a specific field, the performance of the model in question-answering on a specific task can be improved; and by judging the difference between the pre-trained data set and the data set in a specific field, that is, judging the adaptability of the pre-trained model in a specific field, it is possible to pre-judge whether to fine-tune the model in a certain field, which will reduce the consumption of computing resources and time, and avoid over-training the model in unnecessary situations; track and analyze user behavior, evaluate the user's risk level, and perform corresponding risk control; this provides an effective user management mechanism for the intelligent question-answering system, which can prevent malicious behavior.

[0004] However, there are still the following problems in the prior art: In the process of intelligent question answering, in order to obtain undisclosed private data, the questioner transforms the question text in various forms, such as synonym replacement, to conceal the true intention of the question, thereby inducing the language model to output private data, increasing the risk of private data leakage and reducing the reliability of the language model's response. Summary of the invention

[0005] To this end, the present invention provides an AI-based intelligent question and answer data processing system to overcome the problem in the prior art that, in the intelligent question and answer process, in order to obtain undisclosed private data, the questioner converts the question text in various forms to conceal the true intention of the question, thereby inducing the language model to output private data, increasing the risk of private data leakage, and reducing the reliability of the language model's response.

[0006] To achieve the above objectives, the present invention provides an AI-based intelligent question-answering data processing system, which comprises: The permission recording module includes several sensitive information databases for storing sensitive information, and the permission scope of each user terminal for sensitive information; An intention analysis module connected to the permission recording module, used to obtain text data output by the user end, determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each local relevance; an association analysis module, which is connected to the permission recording module and the intention analysis module respectively, and is used to determine the distorted keywords, and calculate the average value of the association between the distorted keywords, the corresponding derivative words and the remaining keywords and the sensitive information outside the permission scope of the user end; A classification module connected to the association analysis module, for determining a semantic sensitivity tendency representation value based on the local association difference and the association degree mean, and classifying the semantic sensitivity tendency category of the user terminal; A processing module, connected to the classification module, is used to process the text data of the user terminal based on the semantic sensitivity tendency category of the user terminal, including: Performing local verification on each text segment of the text data to verify whether the text segment meets the permission standard, generating a number of optimized text segments for the user terminal to select for the text segments that do not meet the permission standard, and transmitting the text segments that meet the verification standard and the optimized text segments selected by the user terminal to the language model; or, transmitting the text data to a language model; A feedback module is used to receive feedback information of the language model to transmit the feedback information to a corresponding user terminal.

[0007] Furthermore, the intention analysis module is used to determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each of the local relevances, including: To determine the semantic collocation degree between a keyword and an adjacent keyword as the local relevance; The variance of each local correlation degree is calculated, and the variance is determined as the local correlation difference.

[0008] Furthermore, the association analysis module is used to determine the misinterpreted keywords and the corresponding derived keywords, including: To generate a number of preliminary screening keywords associated with the distorted keywords; To determine the local correlation between each of the initially screened keywords and the adjacent keywords corresponding to the distorted keywords; Determining the initially screened keywords whose local relevance is greater than a predetermined local relevance threshold as derived keywords; The misinterpreted keywords are keywords whose corresponding local relevance in the text data is less than a predetermined local relevance threshold.

[0009] Furthermore, the association analysis module is used to generate a number of preliminary screening keywords associated with the misinterpreted keywords, including: Invoking a dictionary database to determine synonyms and near synonyms of the misinterpreted keywords as associated primary screening keywords; It is used to determine the keywords with the same pinyin spelling as the distorted keywords as the associated primary screening keywords.

[0010] Furthermore, the classification module is used to determine the semantic sensitivity tendency representation value based on the local association difference and the association degree mean value, including: Using the ratio of the local correlation difference to the local correlation difference threshold as the first semantic sensitive feature; The ratio of the relevance mean to the relevance mean threshold is used as the second semantic sensitive feature; The first semantically sensitive feature and the second semantically sensitive feature are weighted and summed to determine the semantically sensitive tendency representation value.

[0011] Furthermore, the classification module is used to classify the semantic sensitivity tendency categories of the user terminal, including: If the semantic sensitivity tendency representation value is greater than or equal to the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a high sensitivity tendency category; If the semantic sensitivity tendency representation value is less than the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a low sensitivity tendency category.

[0012] Furthermore, the processing module is used to process the text data of the user terminal, including: If the semantic sensitivity tendency of the user terminal is a high sensitivity tendency category, each text segment of the text data is locally verified to verify whether the text segment meets the permission standard, and a number of optimized text segments are generated for the user terminal to select for the text segments that do not meet the permission standard, and the text segments that meet the verification standard and the optimized text segments selected by the user terminal are transmitted to the language model; If the semantic sensitivity tendency of the user terminal is a low sensitivity tendency category, the text data is transmitted to the language model.

[0013] Furthermore, the processing module is used to perform local verification on each text segment of the text data, including: to respectively calculate the relevance between each of the text segments and the standard corpus database; Used to determine whether there are misinterpreted keywords in the text segment.

[0014] Furthermore, the processing module is used to verify whether the text segment meets the authority standard, including: If the text segment meets the permission verification conditions, the text segment is determined to meet the permission standards; The permission verification condition is that the relevance is greater than a predetermined overall relevance threshold and there are no misinterpreted keywords in the text segment.

[0015] Furthermore, the processing module is used to generate a number of optimized text segments, including: Used to generate a number of optimized text segments based on a text rewriting tool; Among them, the correlation between the optimized text segment and the text segment that does not meet the permission standard is greater than or equal to the correlation standard value, and the correlation between the optimized text segment and sensitive information outside the user terminal's permission scope is less than a predetermined sensitive information correlation threshold.

[0016] Compared with the prior art, the present invention sets up a permission recording module, which includes several sensitive information databases for storing sensitive information, and the permission range of each user terminal for sensitive information; an intention analysis module, which is used to obtain text data output by the user terminal, determine the local correlation of keywords relative to adjacent keywords, and determine the local correlation difference of text data based on each local correlation; an association analysis module, which is used to determine the distorted keywords, calculate the average correlation of the distorted keywords, corresponding derivatives and remaining keywords with sensitive information outside the permission range of the user terminal; a classification module, which is used to determine the semantic sensitivity tendency representation value based on the local correlation difference and the correlation mean, and divide the semantic sensitivity tendency category of the user terminal; a processing module, which is used to adaptively process the text data of the user terminal based on the semantic sensitivity tendency category of the user terminal; a feedback module, which is used to receive feedback information of the language model to transmit the feedback information to the corresponding user terminal. The present invention can improve the reliability of the language model reply and avoid the leakage of privacy data.

[0017] In particular, for language models that use a neural network structure, when processing semantic analysis tasks, the input data will be transmitted and transformed through multiple layers of networks. Each layer will perform complex nonlinear mapping on the data and finally obtain the output result. Therefore, complex reasoning may cause the large model to output sensitive content. Furthermore, there may be situations where the language model is induced to output sensitive content through the distortion of question-answering sentences. For example, by abstracting the input text and distorting and replacing the vocabulary in the text, the semantics of some keywords are inclined to sensitive content, inducing the large model to output some content. In actual situations, the semantic collocation degree of each keyword and the adjacent keywords in normal question sentences is relatively uniform. For some sentences that are intentionally induced or have semantic distortion, their local correlation is uneven, and thus the local correlation difference is large. In addition, considering identifying distorted keywords, distorted keywords are keywords with a high tendency of semantic variation, which are likely to cover up the true intention by means of replacement or abstraction. Therefore, considering identifying the derivatives of interval keywords, considering the association between derivatives and sensitive information, and then calculating the semantic sensitivity tendency through local correlation differences and correlation mean, characterizing the tendency of semantic anomalies to induce language models to output sensitive content, providing data support for the subsequent classification of semantic sensitivity tendency categories on the user side, adaptively processing text data, improving the reliability of language model responses, and avoiding privacy data leakage.

[0018] In particular, the present invention divides the semantic sensitivity tendency of the user terminal into categories. For the case where the semantic sensitivity tendency of the user terminal is a high-sensitivity tendency category, each text segment is locally verified, the content within some permissions is retained, and the text segments that do not meet the permission standards are optimized. This can not only avoid directly transmitting the question content involving sensitive information to the language model, but also provide optimized text segments for the user terminal to choose from, considering only the case where there is a semantic deviation, so as to obtain clearer and more accurate question text data, thereby improving the reliability of the language model response and avoiding privacy data leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a functional module diagram of an AI-based intelligent question-answering data processing system according to an embodiment of the invention; Figure 2 A logical decision diagram for classifying semantic sensitivity tendency categories of a user terminal according to an embodiment of the invention; Figure 3 A logical decision diagram for processing text data at a user end according to an embodiment of the invention; Figure 4 A logical decision diagram for verifying whether a text segment meets the permission criteria according to an embodiment of the invention. DETAILED DESCRIPTION

[0020] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0022] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0023] See also Figure 1 As shown, it is a functional module diagram of an AI-based intelligent question and answer data processing system according to an embodiment of the present invention. The AI-based intelligent question and answer data processing system according to an embodiment of the present invention includes: The permission recording module includes several sensitive information databases for storing sensitive information, and the permission scope of each user terminal for sensitive information; An intention analysis module connected to the permission recording module, used to obtain text data output by the user end, determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each local relevance; an association analysis module, which is connected to the permission recording module and the intention analysis module respectively, and is used to determine the distorted keywords, and calculate the average value of the association between the distorted keywords, the corresponding derivative words and the remaining keywords and the sensitive information outside the permission scope of the user end; A classification module connected to the association analysis module, for determining a semantic sensitivity tendency representation value based on the local association difference and the association degree mean, and classifying the semantic sensitivity tendency category of the user terminal; A processing module, connected to the classification module, is used to process the text data of the user terminal based on the semantic sensitivity tendency category of the user terminal, including: Performing local verification on each text segment of the text data to verify whether the text segment meets the permission standard, generating a number of optimized text segments for the user terminal to select for the text segments that do not meet the permission standard, and transmitting the text segments that meet the verification standard and the optimized text segments selected by the user terminal to the language model; or, transmitting the text data to a language model; A feedback module is used to receive feedback information of the language model to transmit the feedback information to a corresponding user terminal.

[0024] It is understandable that in an enterprise, the information involved in each level or department is different. Therefore, the corresponding data that can be obtained by each user terminal is also different. The information of any department involves departmental privacy for other departments, and other departments do not have the authority to obtain it. Therefore, in this embodiment, sensitive information refers to text data that cannot be obtained by users without corresponding permissions, which will not be elaborated here.

[0025] Specifically, there is no limitation on the specific structures of the permission recording module, intent analysis module, association parsing module, classification module, processing module and feedback module, and they themselves or each unit therein can be composed of logical components or a combination of logical components, and the logical components include a field programmable processor, a computer or a microprocessor in a computer.

[0026] Specifically, the intention analysis module is used to determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each of the local relevances, including: To determine the semantic collocation degree between a keyword and an adjacent keyword as the local relevance; The variance of each local correlation degree is calculated, and the variance is determined as the local correlation difference.

[0027] Specifically, there is no limitation on the method of determining the semantic collocation degree between a keyword and an adjacent keyword. In some possible implementations, related words are mapped to a vector space by using word vector models such as Word2Vec and GloVe, and the semantic similarity and collocation possibility are determined by calculating the similarity between vectors, such as cosine similarity. In implementation, cosine similarity can be used to characterize the semantic collocation degree.

[0028] Specifically, the association analysis module is used to determine the misinterpreted keywords and the corresponding derived keywords, including: To generate a number of preliminary screening keywords associated with the distorted keywords; To determine the local correlation between each of the initially screened keywords and the adjacent keywords corresponding to the distorted keywords; Determining the initially screened keywords whose local relevance is greater than a predetermined local relevance threshold as derived keywords; The misinterpreted keywords are keywords whose corresponding local relevance in the text data is less than a predetermined local relevance threshold.

[0029] In this embodiment, the possibility of semantic deviation or misleading intention in the text segment can be represented by misinterpreting the keywords.

[0030] It is understandable that in normal question-and-answer exchanges, there is a certain degree of logical association between the words contained in a complete sentence, the semantic collocation is relatively uniform, and there will be no other words that cause semantic distortion. Therefore, the purpose of setting the local association threshold in this embodiment is to characterize the situation where the meaning expressed by a word and its adjacent words is highly similar. Therefore, the local association threshold is selected within the interval [0.65, 0.72].

[0031] Specifically, the association analysis module is used to generate a number of preliminary screening keywords associated with the misinterpreted keywords, including: Invoking a dictionary database to determine synonyms and near synonyms of the misinterpreted keywords as associated primary screening keywords; It is used to determine the keywords with the same pinyin spelling as the distorted keywords as the associated primary screening keywords.

[0032] Specifically, any open source dictionary database may be used as a basis. The dictionary database stores synonyms and near-synonyms of various words, which will not be described in detail.

[0033] Specifically, the calculation process of the average value of the correlation between the distorted keywords, the corresponding derivative words, and the remaining keywords and the sensitive information outside the user's authority scope includes: The semantic collocation degrees of the distorted keywords, the corresponding derivative words, the remaining keywords and the keywords corresponding to the sensitive information are calculated respectively, and the mean value of the semantic collocation degree is determined as the mean value of the correlation degree.

[0034] Specifically, the classification module is used to determine the semantic sensitivity tendency representation value based on the local association difference and the association degree mean value, including: Using the ratio of the local correlation difference to the local correlation difference threshold as the first semantic sensitive feature; The ratio of the relevance mean to the relevance mean threshold is used as the second semantic sensitive feature; The first semantically sensitive feature and the second semantically sensitive feature are weighted and summed to determine the semantically sensitive tendency representation value.

[0035] In this implementation, when performing weighted summation, the weight of the first semantically sensitive feature is set to 0.45, and the weight of the second semantically sensitive feature is set to 0.55; Specifically, the local correlation difference threshold and the correlation mean threshold are set in this embodiment to indicate that the misinterpreted keywords, the corresponding derivative keywords and the remaining keywords are highly correlated with sensitive information outside the user's authority, and the question content has a high possibility of inducing the acquisition of sensitive information. Therefore, the correlation mean threshold is selected within the interval [0.68, 0.75], and the local correlation difference threshold is determined based on the average value of the local correlation difference; By acquiring a number of question data, extracting local correlation difference data, solving the average value of the local correlation difference, and determining the average value of the local correlation difference as the local correlation difference threshold.

[0036] Specifically, for language models that use a neural network structure, when processing semantic analysis tasks, the input data will be transmitted and transformed through layers of multi-layer networks. Each layer will perform complex nonlinear mapping on the data and finally obtain the output result. Therefore, complex reasoning may cause the large model to output sensitive content. Furthermore, there may be situations where the language model is induced to output sensitive content through the distortion of question and answer sentences. For example, by abstracting the input text and distorting and replacing the vocabulary in the text, the semantics of some keywords are tilted towards sensitive content, inducing the large model to output some content. In actual situations, the semantic matching degree of each keyword and the adjacent keywords in normal question sentences is relatively uniform. For some sentences that are intentionally induced or have semantic distortion, their local correlation is uneven, and thus the local correlation difference is large. In addition, considering the identification of distorted keywords, distorted keywords are keywords with a high tendency of semantic variation, which are likely to cover up the true intention by means of replacement or abstraction. Therefore, considering the identification of derivative words of interval keywords, considering the association between derivative words and sensitive information, and then calculating the semantic sensitivity tendency through local correlation difference and correlation mean, characterizing the tendency of semantic abnormality to induce language model to output sensitive content, providing data support for the subsequent classification of semantic sensitivity tendency categories on the user side, adaptively processing text data, improving the reliability of language model responses, and avoiding privacy data leakage.

[0037] Specifically, see Figure 2 As shown, it is a logical decision diagram for classifying the semantic sensitivity tendency categories of the user terminal according to an embodiment of the present invention. The classification module is used to classify the semantic sensitivity tendency categories of the user terminal, including: If the semantic sensitivity tendency representation value is greater than or equal to the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a high sensitivity tendency category; If the semantic sensitivity tendency representation value is less than the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a low sensitivity tendency category.

[0038] The semantic sensitivity tendency representation threshold is selected in the interval [1.22, 1.36].

[0039] Specifically, see Figure 3 As shown, it is a logical decision diagram for processing text data of the user end in this embodiment. The processing module is used to process the text data of the user end and includes: If the semantic sensitivity tendency of the user terminal is a high sensitivity tendency category, each text segment of the text data is locally verified to verify whether the text segment meets the permission standard, and a number of optimized text segments are generated for the user terminal to select for the text segments that do not meet the permission standard, and the text segments that meet the verification standard and the optimized text segments selected by the user terminal are transmitted to the language model; If the semantic sensitivity tendency of the user terminal is a low sensitivity tendency category, the text data is transmitted to the language model.

[0040] Specifically, the processing module is used to perform local verification on each text segment of the text data, including: to respectively calculate the relevance between each of the text segments and the standard corpus database; Used to determine whether there are misinterpreted keywords in the text segment.

[0041] Specifically, the standard corpus database can be constructed by itself according to specific needs or adopt any existing open source corpus database. The corpus therein are all standard corpora that can be used to train natural language models after screening, which will not be elaborated here.

[0042] Specifically, see Figure 4 As shown, it is a logical decision diagram for verifying whether a text segment meets the permission standard according to an embodiment of the present invention. The processing module is used to verify whether the text segment meets the permission standard, including: If the text segment meets the permission verification conditions, the text segment is determined to meet the permission standards; The permission verification condition is that the relevance is greater than a predetermined overall relevance threshold and there are no misinterpreted keywords in the text segment.

[0043] Specifically, the purpose of setting the overall correlation threshold in this embodiment is to characterize that the correlation between a text segment and its adjacent text segments is high, and the combined text segment group has a high degree of inducing tendency. Therefore, the overall correlation threshold is set in the interval [0.65, 0.78].

[0044] Specifically, the processing module is used to generate a number of optimized text segments, including: Used to generate a number of optimized text segments based on a text rewriting tool; Among them, the correlation between the optimized text segment and the text segment that does not meet the permission standard is greater than or equal to the correlation standard value, and the correlation between the optimized text segment and sensitive information outside the user terminal's permission scope is less than a predetermined sensitive information correlation threshold.

[0045] The standard value of the degree of association is determined according to the overall degree of association threshold, and the standard value of the degree of association is 1.2 times the overall degree of association threshold; The sensitive information correlation threshold is determined based on the correlation standard value, and the sensitive information correlation threshold is 0.85 times the correlation standard value.

[0046] Specifically, the correlation between text segments can also be determined by using the cosine similarity algorithm. After two text segments are vectorized, the cosine similarity is calculated as the correlation, which will not be described in detail.

[0047] Specifically, the present invention divides the semantic sensitivity tendency of the user terminal into categories. When the semantic sensitivity tendency of the user terminal is a highly sensitive tendency category, each text segment is locally verified, the content within some permissions is retained, and the text segments that do not meet the permission standards are optimized. This can not only avoid directly transmitting the question content involving sensitive information to the language model, but also provide optimized text segments for the user terminal to choose from, considering only the case where there is a semantic deviation, so as to obtain clearer and more accurate question text data, thereby improving the reliability of the language model response and avoiding privacy data leakage.

[0048] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An AI-based intelligent question-answering data processing system, characterized in that: include: The permission recording module includes several sensitive information databases for storing sensitive information, and the permission scope of each user terminal for sensitive information; An intention analysis module connected to the permission recording module, used to obtain text data output by the user end, determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each local relevance; an association analysis module, which is connected to the permission recording module and the intention analysis module respectively, and is used to determine the distorted keywords, and calculate the average value of the association between the distorted keywords, the corresponding derivative words and the remaining keywords and the sensitive information outside the permission scope of the user end; A classification module connected to the association analysis module, for determining a semantic sensitivity tendency representation value based on the local association difference and the association degree mean, and classifying the semantic sensitivity tendency category of the user terminal; A processing module, connected to the classification module, is used to process the text data of the user terminal based on the semantic sensitivity tendency category of the user terminal, including: Performing local verification on each text segment of the text data to verify whether the text segment meets the permission standard, generating a number of optimized text segments for the user terminal to select for the text segments that do not meet the permission standard, and transmitting the text segments that meet the verification standard and the optimized text segments selected by the user terminal to the language model; or, transmitting the text data to a language model; A feedback module is used to receive feedback information of the language model to transmit the feedback information to a corresponding user terminal.

2. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The intention analysis module is used to determine the local relevance of a keyword relative to an adjacent keyword, and determine the local relevance difference of the text data based on each of the local relevances, including: To determine the semantic collocation degree between a keyword and an adjacent keyword as the local relevance; The variance of each local correlation degree is calculated, and the variance is determined as the local correlation difference.

3. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The association analysis module is used to determine the misinterpreted keywords and the corresponding derived keywords, including: To generate a number of preliminary screening keywords associated with the distorted keywords; To determine the local correlation between each of the initially screened keywords and the adjacent keywords corresponding to the distorted keywords; Determining the initially screened keywords whose local relevance is greater than a predetermined local relevance threshold as derived keywords; The misinterpreted keywords are keywords whose corresponding local relevance in the text data is less than a predetermined local relevance threshold.

4. The AI-based intelligent question-answering data processing system according to claim 3 is characterized in that: The association analysis module is used to generate a number of preliminary screening keywords associated with the distorted keywords, including: Invoking a dictionary database to determine synonyms and near synonyms of the misinterpreted keywords as associated primary screening keywords; It is used to determine the keywords with the same pinyin spelling as the distorted keywords as the associated primary screening keywords.

5. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The classification module is used to determine the semantic sensitivity tendency representation value based on the local association difference and the association degree mean value, including: Using the ratio of the local correlation difference to the local correlation difference threshold as the first semantic sensitive feature; The ratio of the relevance mean to the relevance mean threshold is used as the second semantic sensitive feature; The first semantically sensitive feature and the second semantically sensitive feature are weighted and summed to determine the semantically sensitive tendency representation value.

6. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The classification module is used to classify the semantic sensitivity tendency categories of the user terminal, including: If the semantic sensitivity tendency representation value is greater than or equal to the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a high sensitivity tendency category; If the semantic sensitivity tendency representation value is less than the semantic sensitivity tendency representation threshold, the semantic sensitivity tendency of the user terminal is determined as a low sensitivity tendency category.

7. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The processing module is used to process the text data of the user terminal, including: If the semantic sensitivity tendency of the user terminal is a high sensitivity tendency category, each text segment of the text data is locally verified to verify whether the text segment meets the permission standard, and a number of optimized text segments are generated for the user terminal to select for the text segments that do not meet the permission standard, and the text segments that meet the verification standard and the optimized text segments selected by the user terminal are transmitted to the language model; If the semantic sensitivity tendency of the user terminal is a low sensitivity tendency category, the text data is transmitted to the language model.

8. The AI-based intelligent question-answering data processing system according to claim 1, characterized in that: The processing module is used to perform local verification on each text segment of the text data, including: to respectively calculate the relevance between each of the text segments and the standard corpus database; Used to determine whether there are misinterpreted keywords in the text segment.

9. The AI-based intelligent question-answering data processing system according to claim 8, characterized in that: The processing module is used to verify whether the text segment meets the authority standard, including: If the text segment meets the permission verification conditions, the text segment is determined to meet the permission standards; The permission verification condition is that the relevance is greater than a predetermined overall relevance threshold and there are no misinterpreted keywords in the text segment.

10. The AI-based intelligent question-answering data processing system according to claim 9, characterized in that: The processing module is used to generate a plurality of optimized text segments, include, Used to generate a number of optimized text segments based on a text rewriting tool; Among them, the correlation between the optimized text segment and the text segment that does not meet the permission standard is greater than or equal to the correlation standard value, and the correlation between the optimized text segment and sensitive information outside the user terminal's permission scope is less than a predetermined sensitive information correlation threshold.

Images