Digital campus intelligent question-answering service method and system based on LLM
By preprocessing and vectorizing campus data, combining Embedding model and LLM model, the problems of low efficiency and low accuracy of traditional question-and-answer systems are solved, and more efficient and accurate knowledge question-and-answer services are achieved to adapt to different scenarios and problem types.
Patent Information
- Application Number
- CN202410324629.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-03-21
AI Technical Summary
Traditional campus knowledge question and answer systems based on rules or templates are inefficient and cannot effectively understand the semantic characteristics of user problems. Large language models are low in accuracy under training a small amount of data, and cannot adapt to different application scenarios and problem types.
By obtaining campus data for preprocessing, segmenting it into text blocks and vectorizing, building an Embedding model for training, generating dense vector data, and using the LLM model for similarity matching search and answer generation, ensuring data privacy and efficiency.
It has improved the accuracy and intelligence level of campus knowledge Q&A system, enhanced the ability to understand and answer generation, adapted to different application scenarios and question types, and effectively improved the overall availability of digital campus smart Q&A services.
Smart Images

Figure CN118194993B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent question-answering technology, and in particular to a digital campus intelligent question-answering service method and system based on LLM. Background Art
[0002] Large Language Model (LLM) refers to a model that can process and generate natural language text. It is based on machine learning and artificial intelligence technology. By learning a large amount of text data, it can generate text with coherence and semantic accuracy. The large language model has good question understanding and answer generation capabilities, can adapt to different application scenarios and question types, and can continuously learn and optimize from a large amount of text data to generate high-quality, coherent, and natural language text. It has strong language generation capabilities and has been widely used in the field of natural language processing. However, the traditional rule-based or template-based keyword retrieval method requires professionals to create a large number of templates and rules, which not only consumes a lot of time and manpower, but also cannot truly understand the semantic characteristics of user questions, resulting in inefficient answers; taking the campus knowledge question-and-answer system as an example, the traditional campus knowledge question-and-answer system requires the design of a large amount of high-quality Q (question, question) & A (answer, answer) sample data, the essence of which is to statistically classify various types of school information, and the technology used is generally based on rule matching or keyword retrieval technology. However, designing and annotating a large number of high-quality Q&A databases requires a lot of manpower and resources, and it is impossible to include all types of questions and campus information. The Q&A database needs to be updated frequently, which is a cumbersome process and inefficient. In addition, the amount of school data is negligible for the amount of data required to train a general large language model. Using very little data to fine-tune the large language model can easily cause the effect of the large language model to decline, resulting in low accuracy of campus knowledge questions and answers. Summary of the invention
[0003] The present application provides a digital campus intelligent question-answering service method based on LLM, which aims to solve the technical problem in the prior art that the accuracy of the question-answering system is low due to the use of a large language model to train a small amount of data.
[0004] This application provides a digital campus intelligent question-answering service method based on LLM, including:
[0005] Acquire campus data, and pre-process the campus data to obtain text data;
[0006] Dividing the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data;
[0007] Build an Embedding model and obtain training data to train the Embedding model;
[0008] Input the first text vector data into the Embedding model for vectorization to obtain second text vector data;
[0009] Obtain a first input question, and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question;
[0010] Performing a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers;
[0011] Input the first input question and multiple matching answers as context into the LLM model, and output answer text information;
[0012] Determining whether feedback information returned according to the answer text information is received within a preset time;
[0013] If feedback information returned according to the answer text information is received, at least one second input question associated with the first input question is obtained, and the second input question is returned as the first input question to the step of inputting the first input question into the Embedding model.
[0014] Preferably, the step of obtaining campus data and preprocessing the campus data to obtain text data includes:
[0015] Acquire campus data, wherein the campus data includes multiple data types and data content corresponding to each data type;
[0016] Determine whether multiple data types and data contents corresponding to the multiple data types meet preset security conditions;
[0017] If any one of the multiple data types and / or the data content corresponding to the data type does not meet the preset security conditions, the data type and / or the data content corresponding to the data type is defined as private data;
[0018] Performing text extraction on the privacy data to obtain a plurality of privacy feature words;
[0019] Each privacy feature word is used as a node to construct a privacy feature graph, wherein adjacent nodes in the privacy feature graph are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one;
[0020] Obtain the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected to the adjacent nodes;
[0021] The privacy value of each node is calculated according to the privacy feature graph, wherein the calculation formula is:
[0022]
[0023] Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node;
[0024] Extract the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature graph, and map the feature value with the corresponding privacy feature word to obtain mapping key information;
[0025] Symmetrically encrypting the mapping key information to obtain encrypted data;
[0026] The encrypted data is added to the corresponding private data, and the private data to which the encrypted data is added is desensitized to obtain text data.
[0027] Preferably, the step of dividing the text data into a plurality of text blocks according to a preset condition comprises:
[0028] Divide the text data into sentences according to preset conditions to obtain multiple language data;
[0029] Traverse all language data and determine whether each language data belongs to the target stack;
[0030] If the language data does not belong to the target stack, the language data is recorded as a segmentation point. After the traversal is completed, multiple segmentation points are obtained.
[0031] Splicing the language data between the segmentation points to obtain multiple spliced texts;
[0032] Traverse all the spliced texts to determine whether the spliced texts and the segmentation points at both ends are positively spliced, and record the segmentation points that are positively spliced as the first segmentation points;
[0033] Splicing the concatenated text with the first segmentation point to obtain a text block;
[0034] After the traversal is completed, multiple text blocks are obtained.
[0035] Preferably, the step of constructing an Embedding model and obtaining training data to train the Embedding model includes:
[0036] Build an Embedding model based on a deep learning framework;
[0037] Set the input and output dimensions of the Embedding model;
[0038] Set the optimizer, loss function, and evaluation metrics of the Embedding model;
[0039] The training data is obtained to train the Embedding model. During the training process, the Embedding model minimizes the loss function according to the output results to complete the training of the Embedding model.
[0040] Preferably, the step of inputting the first text vector data into the Embedding model for vectorization to obtain the second text vector data includes:
[0041] Acquire first text vector data, and establish a digital coding table based on the first text vector data, wherein the first text vector data includes multiple word vectors and context relationship vectors, and the digital coding table includes a mapping relationship between the context relationship vector and the corresponding multiple word vectors;
[0042] Get the input dimension of the Embedding model and build a standard matrix list based on the input dimension;
[0043] Performing dimension matrix conversion on the digital coding table based on the standard matrix list to obtain multiple semantic matrices and multiple semantic matrix lists;
[0044] The multiple semantic matrices are input into the Embedding model in the order of the multiple semantic matrix lists for vectorization, and the second text vector data is output.
[0045] Preferably, the step of performing a similarity matching search on the second text vector data according to the embedded vector to obtain a plurality of matching answers with the highest similarity comprises:
[0046] Calculating multiple cosine values of the embedding vector and the second text vector data based on the cosine similarity principle, wherein the second text vector data includes multiple second text vectors;
[0047] Selecting top k second text vectors with the highest similarity from the second text vector data according to the multiple cosine values as the first answer vector;
[0048] Obtaining a candidate question vector according to the embedding vector;
[0049] Calculating the similarity between the embedding vector and the candidate question vector based on the cosine similarity principle;
[0050] Determining whether the similarity is a preset value;
[0051] If the similarity is a preset value, obtain the top n second answer vectors with the highest similarity corresponding to the candidate question vector, wherein the k value of the top k second text vectors is equal to the n value of the top n second answer vectors;
[0052] The top n second answer vectors and the top k first answer vectors are merged in order of highest similarity to obtain multiple matching answers.
[0053] Preferably, the step of obtaining at least one second input question related to the first input question comprises:
[0054] Obtain at least one keyword and at least one context word in the first input question;
[0055] Obtaining a key vector corresponding to at least one of the keywords;
[0056] Obtaining a context vector corresponding to at least one of the context words;
[0057] The association value is calculated according to the key vector and the context vector, wherein the calculation formula is:
[0058]
[0059] Wherein, S represents the association value, f(a) represents the key vector, f(A) represents the weight corresponding to the key vector, f(b) represents the context vector, and f(B) represents the weight corresponding to the context vector;
[0060] Obtaining a synonym vector corresponding to the associated value;
[0061] A question vector is generated according to the synonym vector and the context vector, and a second input question is generated according to the question vector.
[0062] The present application also provides a digital campus intelligent question-answering service system based on LLM, including:
[0063] A preprocessing module is used to obtain campus data and preprocess the campus data to obtain text data;
[0064] A segmentation module, used for segmenting the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data;
[0065] The construction module is used to build the Embedding model and obtain training data to train the Embedding model;
[0066] A vectorization module, used for inputting the first text vector data into the Embedding model for vectorization to obtain second text vector data;
[0067] An acquisition module, configured to acquire a first input question and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question;
[0068] A similarity matching module, used to perform a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers;
[0069] An LLM model module, used for inputting a first input question and a plurality of matching answers as context into an LLM model, and outputting answer text information;
[0070] A judgment module, used to judge whether the feedback information returned according to the answer text information is received within a preset time;
[0071] A feedback module is used to obtain at least one second input question associated with the first input question if feedback information returned based on the answer text information is received, and return the second input question as the first input question to the step of inputting the first input question into the Embedding model.
[0072] Preferably, the preprocessing module comprises:
[0073] A first data acquisition unit is used to acquire campus data, wherein the campus data includes multiple data types and data content corresponding to each data type;
[0074] A judging unit, used to judge whether a plurality of data types and data contents corresponding to the plurality of data types meet preset security conditions;
[0075] If any one of the multiple data types and / or the data content corresponding to the data type does not meet the preset security conditions, the data type and / or the data content corresponding to the data type is defined as private data;
[0076] A text extraction unit, used to perform text extraction on the privacy data to obtain a plurality of privacy feature words;
[0077] A construction unit, used to construct a privacy feature map using each privacy feature word as a node, wherein adjacent nodes in the privacy feature map are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one;
[0078] The second acquisition unit is used to obtain the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected with the adjacent node;
[0079] A calculation unit is used to calculate the privacy value of each node according to the privacy feature graph, wherein the calculation formula is:
[0080]
[0081] Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node;
[0082] An extraction unit, used to extract the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature map, and map the feature value with the corresponding privacy feature word to obtain mapping key information;
[0083] An encryption unit, used for symmetrically encrypting the mapping key information to obtain encrypted data;
[0084] The desensitizing unit is used to add the encrypted data to the corresponding private data, and perform desensitization processing on the private data to which the encrypted data is added, so as to obtain text data.
[0085] Preferably, the segmentation module comprises:
[0086] A division unit, used for dividing the text data into sentences according to preset conditions to obtain multiple language data;
[0087] The first traversal unit is used to traverse all language data and determine whether each language data belongs to the target stack;
[0088] If the language data does not belong to the target stack, the language data is recorded as a segmentation point. After the traversal is completed, multiple segmentation points are obtained.
[0089] A concatenation unit, used for concatenating the language data between the segmentation points to obtain a plurality of concatenated texts;
[0090] The second traversal unit is used to traverse all the spliced texts, determine whether the spliced texts and the segmentation points at both ends are positively spliced, and record the segmentation points that are positively spliced as the first segmentation points;
[0091] The second splicing unit is used to splice the spliced text with the first segmentation point to obtain a text block. After the traversal is completed, multiple text blocks are obtained.
[0092] The beneficial effects of the present application are as follows: after obtaining campus data, the campus data is first preprocessed, so that the privacy leakage caused by the feature recording and learning of the real data by the generative large language model can be eliminated from the source; after the preprocessing is completed, the text data obtained based on the preprocessing is segmented into multiple text blocks, and the text blocks are vectorized, which can be directly converted into sparse vectors to facilitate the subsequent data processing; an Embedding model is constructed, and training data is obtained to train the Embedding model, and the first text vector data is input into the Embedding model for vectorization to obtain the second text vector data, which can reduce the dimension of the second text vector and make the generated second text vector data dense, thereby And it can improve the comprehensive information capability of the second text vector data; when the user needs to make a query, the embedding vector corresponding to the first input question can be obtained, and multiple matching answers that are most similar to the embedded vector can be matched from the second text vector data based on similarity calculation, and then the first input question and multiple matching answers are input into the LLM model as context to output the answer text information. Compared with directly inputting campus data into the large language model to output the answer text information, this application improves the overall availability of school data, has better question understanding and answer generation capabilities, can adapt to different application scenarios and question types, can effectively improve the accuracy and intelligence level of the campus life knowledge question and answer system, and to a certain extent improves the accuracy of the digital campus intelligent question and answer service. BRIEF DESCRIPTION OF THE DRAWINGS
[0093] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present application.
[0094] Figure 2 A schematic diagram of the system structure of an embodiment of the present application.
[0095] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0096] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0097] like Figure 1 , Figure 2 As shown, the present application provides a digital campus intelligent question-answering service method based on LLM, including:
[0098] S1. Obtain campus data and pre-process the campus data to obtain text data;
[0099] S2, dividing the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data;
[0100] S3. Build an Embedding model and obtain training data to train the Embedding model;
[0101] S4, inputting the first text vector data into the Embedding model for vectorization to obtain second text vector data;
[0102] S5. Obtain a first input question, and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question;
[0103] S6, performing a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers;
[0104] S7, inputting the first input question and the multiple matching answers as context into the LLM model, and outputting answer text information;
[0105] S8, determining whether feedback information returned according to the answer text information is received within a preset time;
[0106] S9. If feedback information returned based on the answer text information is received, obtain at least one second input question associated with the first input question, and return the second input question as the first input question to the step of inputting the first input question into the Embedding model.
[0107] As described in the above steps S1-S9, traditional campus knowledge question and answer systems in the prior art require the design of a large amount of high-quality question and answer sample data, the essence of which is to statistically classify various types of school information. The technology used is generally rule-based matching or keyword retrieval technology. However, the traditional rule-based or template-based keyword retrieval method requires professionals to create a large number of templates and rules, which not only consumes a lot of time and manpower, but also fails to truly understand the semantic characteristics of user questions, resulting in inefficient answers, and there is no way to solve the problem of rapid and continuous updating of campus knowledge, and the portability is also poor.Deep learning methods require that the processed text is continuous, while campus data is usually of many types and discontinuous. In addition, the models corresponding to deep learning methods usually require a large amount of labeled data to train, and large-scale labeled data is very expensive and time-consuming. The scarcity of labeled data limits the performance of deep learning methods. Some researchers have proposed to deploy a question-answering system locally by training a large language model or directly using private data to fine-tune the large language model. However, this requires a lot of manpower and material resources, and cannot include all types of questions. The question-answering database needs to be updated frequently, and the process is cumbersome and inefficient. In addition, the data of each school is different, and the amount of data of a single school is relatively small compared to the large language model. Therefore, training with a small amount of data leads to a low accuracy rate of the question-answering system. Based on this, the present application first obtains campus data. Campus data can include resource data such as student handbooks, central knowledge bases, graduate student handbooks, and teacher-student information. Therefore, there is privacy data in the campus data. Since the campus data contains privacy data, if the privacy data is directly input into the large language model, certain data features will be retained in the large language model, causing data leakage. Therefore, after obtaining the campus data, the present application first pre-processes the campus data, so that it can be obtained from The privacy leakage caused by the feature recording and learning of real data by the generative large language model is eliminated at the source; after the preprocessing is completed, the text data obtained based on the preprocessing is segmented into multiple text blocks, and vectorized, and the first text vector data obtained by vectorization is stored in the text vector database of the LangChain framework. Specifically, the text data can be segmented based on the text segmenter. Compared with the prior art that uses, for example, a fine-tuning model for segmentation, the use of a text segmenter does not require training of text data, and new content can be added in real time, and there is no need to train once every time new content is added. For campus data that often needs to update content, using a text segmenter for segmentation and vectorizing it can improve its segmentation efficiency and reduce segmentation costs. By vectorizing the text block, it can be directly converted into a sparse vector, which is convenient for later data processing; then an Embedding model can be constructed, and training data can be obtained to train the Embedding model. The Embedding model is a commonly used technology in the fields of machine learning and natural language processing. It aims to map high-dimensional data to a low-dimensional space.By inputting the first text vector data into the Embedding model for vectorization to obtain the second text vector data, the dimension of the second text vector can be reduced. The Embedding model is good at processing high-dimensional sparse vectors. While the Embedding model reduces the dimension and prevents parameter explosion, it can also realize the densification of sparse vectors. Therefore, for campus data with a small amount of data, the Embedding model is given priority. This can make the generated second text vector data dense, thereby improving the comprehensive information capacity of the second text vector data; when the user needs to make a query, the first input question can be obtained and the embedding vector corresponding to the first input question can be obtained through the Embedding model. Based on the similarity calculation, multiple matching answers that are most similar to the embedded vector are matched from the second text vector data, and then the first input question and multiple matching answers are input as context into the LLM model to output the answer text information. After generating the answer text information, it does not exit directly. This application also determines whether the feedback information returned according to the answer text information is received within the preset time. The user's satisfaction can be understood through the feedback information. If the feedback information is that the user is dissatisfied, at least one second input question associated with the first input question is obtained, and the second input question is returned as the first input question to the step of inputting the first input question into the Embedding model until the feedback information satisfies the user. In this way, compared with directly inputting campus data into the large language model to output the answer text information, this application improves the overall availability of school data. Compared with the existing campus knowledge question and answer system solutions known from the survey, this application has better problem understanding and answer generation capabilities, can adapt to different application scenarios and question types, and can effectively improve the accuracy and intelligence level of the campus life knowledge question and answer system, and to a certain extent improve the accuracy of the digital campus intelligent question and answer service. It should be noted that the steps in this application are not only applicable to campus data, but also to a small amount of data such as hospital data and company data, which are not limited here.
[0108] In one embodiment, the step S1 of acquiring campus data and preprocessing the campus data to obtain text data includes:
[0109] S11, acquiring campus data, wherein the campus data includes multiple data types and data content corresponding to each data type;
[0110] S12, determining whether multiple data types and data contents corresponding to the multiple data types meet preset security conditions;
[0111] S13. If the multiple data types and any one of the multiple data types and / or the data content corresponding to the data type do not meet the preset security conditions, define the data type and / or the data content corresponding to the data type as private data;
[0112] S14, performing text extraction on the privacy data to obtain a plurality of privacy feature words;
[0113] S15, constructing a privacy feature graph using each privacy feature word as a node, wherein adjacent nodes in the privacy feature graph are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one;
[0114] S16, obtaining the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected to the adjacent node;
[0115] S17. Calculate the privacy value of each node according to the privacy feature graph, where the calculation formula is:
[0116]
[0117] Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node;
[0118] S18. Extracting the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature map, and mapping the feature value with the corresponding privacy feature word to obtain mapping key information;
[0119] S19, symmetrically encrypting the mapping key information to obtain encrypted data;
[0120] S191. Add the encrypted data to the corresponding private data, and perform desensitization processing on the private data to which the encrypted data is added, to obtain text data.
[0121] As described in the above steps S11-S191, since the campus data includes privacy data such as teacher-student data, research data, etc. in addition to knowledge data, in order to better protect them and prevent some privacy data from remaining in subsequent models, this embodiment, after obtaining the campus data, performs privacy judgment on the data type and corresponding data content in the campus data, thereby filtering out the privacy data from the campus data, rather than protecting the privacy of the entire campus. This can reduce the complexity of the calculation, and only protects the privacy data, which can also improve the authenticity of the campus data as a whole, and facilitate the improvement of the answer text information generated later. accuracy; specifically, after filtering out the privacy data, the privacy feature words in the privacy data are extracted, and a privacy feature map is constructed based on multiple privacy feature words. In the privacy feature map, each privacy feature word is taken as a node and numbered, and the relationship between nodes is connected through connecting lines. In this way, the importance of the privacy feature word in the data content can be judged by the connecting lines owned by each node, so as to obtain the corresponding desensitization prediction loss value based on the number of connecting lines. The desensitization prediction loss value indicates the loss accuracy caused by the desensitization of the privacy feature word; and in some data content, a single The privacy level represented by the privacy feature word is relatively low, but when combined with other privacy feature words, its privacy level will be improved. Therefore, in addition to considering the privacy impact brought by a single privacy feature word, this embodiment also considers the privacy impact brought by the combination of privacy feature words and privacy feature words. Therefore, it is also necessary to obtain the associated weight value of each node connected to the adjacent node. By obtaining the associated weight value, the privacy level brought by its combination can be understood, and the privacy value of each node is calculated based on the privacy feature map, the desensitization prediction loss value and the associated weight value. The privacy value calculated in this way can be expressed more accurately. After calculating the privacy degree of the privacy feature word and the privacy value of each privacy feature word, the maximum privacy value is selected as the feature value of the privacy feature map, and the feature value is mapped with the corresponding privacy feature word to obtain the mapping key information, encrypt the mapping key information, and add it to the corresponding privacy data, and then desensitize the entire privacy data to obtain text data, so that the value with the highest privacy degree in the privacy data can be found and encrypted separately, and then desensitized with other privacy data, so as to deepen the privacy protection of privacy feature words with higher privacy degrees and reduce the risk of information leakage. It should be noted that the above-mentioned steps are taken when only any one data type and / or the data content corresponding to the data type is privacy data, so it only corresponds to the generation of one privacy feature map. When there are multiple privacy data, multiple privacy feature maps are generated, and the process is consistent with the above, which will not be repeated here.
[0122] In one embodiment, the step S2 of dividing the text data into a plurality of text blocks according to a preset condition comprises:
[0123] S21, dividing the text data into sentences according to preset conditions to obtain multiple language data;
[0124] S22, traverse all language data, and determine whether each language data belongs to the target stack;
[0125] S23, if the language data does not belong to the target stack, the language data is recorded as a segmentation point, and after the traversal is completed, multiple segmentation points are obtained;
[0126] S24, concatenating the language data between the segmentation points to obtain a plurality of concatenated texts;
[0127] S25, traversing all the spliced texts, determining whether the spliced texts and the segmentation points at both ends are positively spliced, and recording the segmentation points that are positively spliced as the first segmentation points;
[0128] S26, splicing the spliced text with the first segmentation point to obtain a text block;
[0129] S27. After the traversal is completed, multiple text blocks are obtained.
[0130] As described in the above steps S21-S27, when the text data is segmented, the text data is first divided into sentences according to preset conditions, wherein the preset conditions may be segmented by paragraphs, sentences, chapters, words or other, and the segmentation may be performed according to the needs, and no unique restriction is made here; after the segmentation, a plurality of language data are obtained, and it is determined whether each language data belongs to a target stack, that is, a target topic; if not, the language data is recorded as a segmentation point, and after traversing all the language data, a plurality of segmentation points are obtained, and the language data between the segmentation points are spliced to obtain a plurality of spliced texts. After the segmentation is completed, the present embodiment determines whether the spliced text and the segmentation points at both ends are forward spliced, and the forward splicing means that the segmentation point and the spliced text belong to the same target topic, and the spliced text is spliced with the first segmentation point to obtain a text block, which can avoid splicing the segmentation point with other types of topics and improve the accuracy of the segmentation.
[0131] In one embodiment, the step S3 of building an Embedding model and acquiring training data to train the Embedding model includes:
[0132] S31. Build an Embedding model based on a deep learning framework;
[0133] S32. Set the input dimension and output dimension of the Embedding model;
[0134] S33. Set the optimizer, loss function and evaluation index of the Embedding model;
[0135] S34. Obtain training data to train the Embedding model. During the training process, the Embedding model minimizes the loss function according to the output results to complete the training of the Embedding model.
[0136] As described in the above steps S31-S34, by constructing the Embedding model, it is convenient to convert the campus data into a low dimension. When training the Embedding model, the training data is obtained to train it, rather than using the campus data directly for training. This can avoid leaving data residues in the campus data during the training process. The training data can be approximate data of the campus data. When constructing the Embedding model, different required input dimensions and output dimensions, optimizers, loss functions, and evaluation indicators can be set according to the data type. No unique limitation is made here.
[0137] In one embodiment, the step S4 of inputting the first text vector data into the Embedding model for vectorization to obtain the second text vector data includes:
[0138] S41, obtaining first text vector data, and establishing a digital coding table based on the first text vector data, wherein the first text vector data includes a plurality of word vectors and a context relationship vector, and the digital coding table includes a mapping relationship between the context relationship vector and the corresponding plurality of word vectors;
[0139] S42. Obtain the input dimension of the Embedding model and construct a standard matrix list according to the input dimension;
[0140] S43, performing dimension matrix conversion on the digital coding table based on the standard matrix list to obtain multiple semantic matrices and multiple semantic matrix lists;
[0141] S44. Input the multiple semantic matrices into the Embedding model in the order of the multiple semantic matrix lists for vectorization, and output the second text vector data.
[0142] As described in the above steps S41-S44, before the first text vector data is input into the Embedding model, in order to better understand the semantics, a digital coding table can be established based on the first text vector data, that is, the first text vector data is represented in the form of a digital coding table, and the input dimension of the Embedding model is obtained, so as to construct a standard matrix list based on the input dimension, and perform dimension matrix conversion on the digital coding table based on the standard matrix list, so as to obtain multiple semantic matrices and multiple semantic matrix lists. If the first text vector data is directly converted into a matrix list, it is easy to ignore the contextual relationship between word vectors and word vectors, resulting in semantic missing information. Therefore, by establishing a digital coding table, the mapping relationship between each vector can be understood from the table, and then matrix conversion is performed according to the digital coding table, and the multiple semantic matrices are input into the Embedding model in the order of the converted multiple semantic matrix lists for vectorization to obtain the second text vector data. In this way, the generated second text vector data can express the semantics of campus data as accurately, completely and objectively as possible.
[0143] In one embodiment, the step S6 of performing a similarity matching search on the second text vector data according to the embedded vector to obtain a plurality of matching answers with the highest similarity includes:
[0144] S61, calculating multiple cosine values of the embedded vector and the second text vector data based on the cosine similarity principle, wherein the second text vector data includes multiple second text vectors;
[0145] S62, selecting top k second text vectors with the highest similarity from the second text vector data according to the multiple cosine values as the first answer vector;
[0146] S63, obtaining a candidate question vector according to the embedding vector;
[0147] S64, calculating the similarity between the embedding vector and the candidate question vector based on the cosine similarity principle;
[0148] S65, determining whether the similarity is a preset value;
[0149] S66, if the similarity is a preset value, obtain the top n second answer vectors with the highest similarity corresponding to the candidate question vector, wherein the k value of the top k second text vectors is equal to the n value of the top n second answer vectors;
[0150] S67 , merge the top n second answer vectors and the top k first answer vectors in order of highest similarity to obtain multiple matching answers.
[0151] As described in the above steps S61-S67, the cosine similarity is used to calculate the similarity between the embedded vector and the second text vector data, that is, the cosine value between the two vectors is calculated, ranging from -1 to 1, 1 represents complete similarity, -1 represents complete opposite, and 0 represents no similarity. According to the value of the similarity metric, comparison and evaluation are performed. The larger the similarity metric, the higher the similarity between the objects. Therefore, the top k text vectors most similar to the embedded vector can be matched from the second text vector data through similarity calculation as the first answer vector, wherein the value range of k can be customized and is not limited here; since campus data is frequently updated, new second text vectors are often added. In order to answer the question more completely, after obtaining the first answer vector, this embodiment obtains an alternative question vector based on the embedded vector, and calculates the similarity between the two based on the cosine similarity. If the similarity is a preset value, the top n second answer vectors with the highest similarity corresponding to the alternative question vector are obtained, wherein the value range of n is consistent with k, and the top n second answer vectors and the top k first answer vectors are merged in order of the highest similarity, that is, top The second answer vector of n=1 is merged with the first answer vector of top k=1, and the second answer vector of top n=2 is merged with the first answer vector of top k=2 to obtain multiple matching answers, which can supplement the matching answers and make their answers more complete.
[0152] In one embodiment, the step S9 of obtaining at least one second input question related to the first input question includes:
[0153] S91, obtaining at least one keyword and at least one context word in the first input question;
[0154] S92, obtaining a key vector corresponding to at least one of the keywords;
[0155] S93, obtaining a context vector corresponding to at least one of the context words;
[0156] S94. Calculate an association value according to the key vector and the context vector, wherein the calculation formula is:
[0157]
[0158] Wherein, S represents the association value, f(a) represents the key vector, f(A) represents the weight corresponding to the key vector, f(b) represents the context vector, and f(B) represents the weight corresponding to the context vector;
[0159] S95, obtaining a synonym vector corresponding to the association value;
[0160] S96. Generate a question vector according to the synonym vector and the context vector, and generate a second input question according to the question vector.
[0161] As described in the above steps S91-S96, by obtaining at least one keyword and at least one context word in the first input question, the correlation value between the two can be calculated based on the corresponding key vector and context vector. Calculating the correlation value helps to understand the rationality of the keyword in the context, so that the corresponding synonym vector can be obtained according to the correlation value, so that the synonym vector logically conforms to the first input question, which helps to make the generated second input question more accurate and more consistent with the first input question.
[0162] The present application also provides a digital campus intelligent question-answering service system based on LLM, including:
[0163] The preprocessing module 1 is used to obtain campus data and preprocess the campus data to obtain text data;
[0164] A segmentation module 2, used for segmenting the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data;
[0165] Building module 3, used to build the Embedding model and obtain training data to train the Embedding model;
[0166] A vectorization module 4 is used to input the first text vector data into the Embedding model for vectorization to obtain second text vector data;
[0167] An acquisition module 5 is used to acquire a first input question and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question;
[0168] A similarity matching module 6 is used to perform a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers;
[0169] LLM model module 7, used for inputting the first input question and multiple matching answers as context into the LLM model, and outputting answer text information;
[0170] A determination module 8 is used to determine whether feedback information returned according to the answer text information is received within a preset time;
[0171] Feedback module 9 is used to obtain at least one second input question associated with the first input question if feedback information returned based on the answer text information is received, and return the second input question as the first input question to the step of inputting the first input question into the Embedding model.
[0172] In one embodiment, the preprocessing module 1 comprises:
[0173] A first data acquisition unit is used to acquire campus data, wherein the campus data includes multiple data types and data content corresponding to each data type;
[0174] A judging unit, used to judge whether a plurality of data types and data contents corresponding to the plurality of data types meet preset security conditions;
[0175] If any one of the multiple data types and / or the data content corresponding to the data type does not meet the preset security conditions, the data type and / or the data content corresponding to the data type is defined as private data;
[0176] A text extraction unit, used to perform text extraction on the privacy data to obtain a plurality of privacy feature words;
[0177] A construction unit, used to construct a privacy feature map using each privacy feature word as a node, wherein adjacent nodes in the privacy feature map are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one;
[0178] The second acquisition unit is used to obtain the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected with the adjacent node;
[0179] A calculation unit is used to calculate the privacy value of each node according to the privacy feature graph, wherein the calculation formula is:
[0180]
[0181] Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node;
[0182] An extraction unit, used to extract the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature map, and map the feature value with the corresponding privacy feature word to obtain mapping key information;
[0183] An encryption unit, used for symmetrically encrypting the mapping key information to obtain encrypted data;
[0184] The desensitizing unit is used to add the encrypted data to the corresponding private data, and perform desensitization processing on the private data to which the encrypted data is added, so as to obtain text data.
[0185] In one embodiment, the segmentation module 2 comprises:
[0186] A division unit, used for dividing the text data into sentences according to preset conditions to obtain multiple language data;
[0187] The first traversal unit is used to traverse all language data and determine whether each language data belongs to the target stack;
[0188] If the language data does not belong to the target stack, the language data is recorded as a segmentation point. After the traversal is completed, multiple segmentation points are obtained.
[0189] A concatenation unit, used for concatenating the language data between the segmentation points to obtain a plurality of concatenated texts;
[0190] The second traversal unit is used to traverse all the spliced texts, determine whether the spliced texts and the segmentation points at both ends are positively spliced, and record the segmentation points that are positively spliced as the first segmentation points;
[0191] The second splicing unit is used to splice the spliced text with the first segmentation point to obtain a text block. After the traversal is completed, multiple text blocks are obtained.
[0192] It should be noted that each module and unit in the LLM-based digital campus intelligent question and answer service system corresponds to the steps in the LLM-based digital campus intelligent question and answer service method.
[0193] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.
[0194] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A digital campus intelligent question-answering service method based on LLM, characterized in that: include: Acquire campus data, and pre-process the campus data to obtain text data; Dividing the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data; Build an Embedding model and obtain training data to train the Embedding model; Input the first text vector data into the Embedding model for vectorization to obtain second text vector data; Obtain a first input question, and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question; Performing a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers; Input the first input question and multiple matching answers as context into the LLM model, and output answer text information; Determining whether feedback information returned according to the answer text information is received within a preset time; If feedback information returned according to the answer text information is received, obtaining at least one second input question associated with the first input question, and returning the second input question as the first input question to the step of inputting the first input question into the Embedding model; The step of obtaining campus data and preprocessing the campus data to obtain text data includes: Acquire campus data, wherein the campus data includes multiple data types and data content corresponding to each data type; Determine whether multiple data types and data contents corresponding to the multiple data types meet preset security conditions; If any one of the multiple data types and / or the data content corresponding to the data type does not meet the preset security conditions, the data type and / or the data content corresponding to the data type is defined as private data; Performing text extraction on the privacy data to obtain a plurality of privacy feature words; Each privacy feature word is used as a node to construct a privacy feature graph, wherein adjacent nodes in the privacy feature graph are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one; Obtain the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected to the adjacent nodes; The privacy value of each node is calculated according to the privacy feature graph, wherein the calculation formula is: Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node; Extract the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature graph, and map the feature value with the corresponding privacy feature word to obtain mapping key information; Symmetrically encrypting the mapping key information to obtain encrypted data; The encrypted data is added to the corresponding private data, and the private data to which the encrypted data is added is desensitized to obtain text data.
2. The LLM-based digital campus intelligent question-answering service method according to claim 1 is characterized in that: The step of dividing the text data into a plurality of text blocks according to the preset conditions comprises: Divide the text data into sentences according to preset conditions to obtain multiple language data; Traverse all language data and determine whether each language data belongs to the target stack; If the language data does not belong to the target stack, the language data is recorded as a segmentation point. After the traversal is completed, multiple segmentation points are obtained. Splicing the language data between the segmentation points to obtain multiple spliced texts; Traverse all the spliced texts to determine whether the spliced texts and the segmentation points at both ends are positively spliced, and record the segmentation points that are positively spliced as the first segmentation points; Splicing the concatenated text with the first segmentation point to obtain a text block; After the traversal is completed, multiple text blocks are obtained.
3. The LLM-based digital campus intelligent question-answering service method according to claim 1 is characterized in that: The steps of constructing the Embedding model and obtaining training data to train the Embedding model include: Build an Embedding model based on a deep learning framework; Set the input and output dimensions of the Embedding model; Set the optimizer, loss function, and evaluation metrics of the Embedding model; The training data is obtained to train the Embedding model. During the training process, the Embedding model minimizes the loss function according to the output results to complete the training of the Embedding model.
4. The LLM-based digital campus intelligent question-answering service method according to claim 1, characterized in that: The step of inputting the first text vector data into the Embedding model for vectorization to obtain the second text vector data includes: Acquire first text vector data, and establish a digital coding table based on the first text vector data, wherein the first text vector data includes multiple word vectors and context relationship vectors, and the digital coding table includes a mapping relationship between the context relationship vector and the corresponding multiple word vectors; Get the input dimension of the Embedding model and build a standard matrix list based on the input dimension; Performing dimension matrix conversion on the digital coding table based on the standard matrix list to obtain multiple semantic matrices and multiple semantic matrix lists; The multiple semantic matrices are input into the Embedding model in the order of the multiple semantic matrix lists for vectorization, and the second text vector data is output.
5. The LLM-based digital campus intelligent question-answering service method according to claim 1 is characterized in that: The step of performing a similarity matching search on the second text vector data according to the embedded vector to obtain a plurality of matching answers with the highest similarity comprises: Calculating multiple cosine values of the embedding vector and the second text vector data based on the cosine similarity principle, wherein the second text vector data includes multiple second text vectors; Selecting top k second text vectors with the highest similarity from the second text vector data according to the multiple cosine values as the first answer vector; Obtaining a candidate question vector according to the embedding vector; Calculating the similarity between the embedding vector and the candidate question vector based on the cosine similarity principle; Determining whether the similarity is a preset value; If the similarity is a preset value, obtain the top n second answer vectors with the highest similarity corresponding to the candidate question vector, wherein the k value of the top k second text vectors is equal to the n value of the top n second answer vectors; The top n second answer vectors and the top k first answer vectors are merged in order of highest similarity to obtain multiple matching answers.
6. The LLM-based digital campus intelligent question-answering service method according to claim 1 is characterized in that: The step of obtaining at least one second input question related to the first input question comprises: Obtain at least one keyword and at least one context word in the first input question; Obtaining a key vector corresponding to at least one of the keywords; Obtaining a context vector corresponding to at least one of the context words; The association value is calculated according to the key vector and the context vector, wherein the calculation formula is: Wherein, S represents the association value, f(a) represents the key vector, f(A) represents the weight corresponding to the key vector, f(b) represents the context vector, and f(B) represents the weight corresponding to the context vector; Obtaining a synonym vector corresponding to the associated value; A question vector is generated according to the synonym vector and the context vector, and a second input question is generated according to the question vector.
7. A digital campus intelligent question-answering service system based on LLM, characterized in that: include: A preprocessing module is used to obtain campus data and preprocess the campus data to obtain text data; A segmentation module, used for segmenting the text data into a plurality of text blocks according to a preset condition, and vectorizing the plurality of text blocks to obtain first text vector data; The construction module is used to build the Embedding model and obtain training data to train the Embedding model; A vectorization module, used for inputting the first text vector data into the Embedding model for vectorization to obtain second text vector data; An acquisition module, configured to acquire a first input question and input the first input question into an Embedding model to generate an embedding vector corresponding to the first input question; A similarity matching module, used to perform a similarity matching search on the second text vector data according to the embedded vector to obtain multiple matching answers; An LLM model module, used for inputting a first input question and a plurality of matching answers as context into an LLM model, and outputting answer text information; A judgment module, used to judge whether the feedback information returned according to the answer text information is received within a preset time; A feedback module, configured to, upon receiving feedback information returned according to the answer text information, obtain at least one second input question associated with the first input question, and return the second input question as the first input question to the step of inputting the first input question into the Embedding model; The preprocessing module comprises: A first data acquisition unit is used to acquire campus data, wherein the campus data includes multiple data types and data content corresponding to each data type; A judging unit, used to judge whether a plurality of data types and data contents corresponding to the plurality of data types meet preset security conditions; If any one of the multiple data types and / or the data content corresponding to the data type does not meet the preset security conditions, the data type and / or the data content corresponding to the data type is defined as private data; A text extraction unit, used to perform text extraction on the privacy data to obtain a plurality of privacy feature words; A construction unit, used to construct a privacy feature map using each privacy feature word as a node, wherein adjacent nodes in the privacy feature map are connected by connecting lines, and if there are identical privacy feature words, the nodes corresponding to the identical privacy feature words are recorded as one; The second acquisition unit is used to obtain the desensitization prediction loss value corresponding to each node and the associated weight value of each node connected with the adjacent node; A calculation unit is used to calculate the privacy value of each node according to the privacy feature graph, wherein the calculation formula is: Among them, the R i represents the privacy value of the i-th node, and A represents all nodes pointing to R i The collection of nodes, x i,j represents the associated weight value of the connection between the i-th node and the j-th node, x i,k represents the association weight value of the connection between the i-th node and the k-th node, B represents the set of nodes connected by the node Ri, and S i represents the ranking value of the i-th node in the privacy feature graph, L i Represents the desensitization prediction loss value corresponding to the i-th node; An extraction unit, used to extract the maximum value from the privacy values of multiple nodes as the feature value of the privacy feature map, and map the feature value with the corresponding privacy feature word to obtain mapping key information; An encryption unit, used for symmetrically encrypting the mapping key information to obtain encrypted data; The desensitizing unit is used to add the encrypted data to the corresponding private data, and perform desensitization processing on the private data to which the encrypted data is added, so as to obtain text data.
8. The LLM-based digital campus intelligent question-answering service system according to claim 7 is characterized in that: The segmentation module comprises: A division unit, used for dividing the text data into sentences according to preset conditions to obtain multiple language data; The first traversal unit is used to traverse all language data and determine whether each language data belongs to the target stack; If the language data does not belong to the target stack, the language data is recorded as a segmentation point. After the traversal is completed, multiple segmentation points are obtained. A concatenation unit, used for concatenating the language data between the segmentation points to obtain a plurality of concatenated texts; The second traversal unit is used to traverse all the spliced texts, determine whether the spliced texts and the segmentation points at both ends are positively spliced, and record the segmentation points that are positively spliced as the first segmentation points; The second splicing unit is used to splice the spliced text with the first segmentation point to obtain a text block. After the traversal is completed, multiple text blocks are obtained.
Citation Information
Patent Citations
Intelligent question-answering system and method suitable for vertical field and application of intelligent question-answering system and method
CN116805001A
