A college admissions consultation question-answering system based on an offline general large language model

Abstract

The present invention provides a college admissions consultation question-and-answer system based on an offline universal large language model. The system comprises: an input module for inputting college admissions consultation question text; a classification and retrieval model for classifying and predicting the question text to obtain a predicted classification label; screening various text databases for corresponding classifications; performing feature extraction on the question text and each document in the corresponding classification text database, performing similarity matching, and selecting the document text with the highest similarity as the output result; a combination module for organizing the question text and the output document text using prompts to form an organized text; and an offline universal large language model for taking the organized text as input and outputting college admissions consultation response data. This system utilizes the offline universal large language model to replace the answer extraction module of traditional question-and-answer systems, reducing model training costs while making the question-and-answer system more intelligent.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically, to a college admissions consultation question-answering system based on an offline universal large language model. Background Art

[0002] As the amount of information involved in question-answering systems continues to increase and the answers become more general, question-answering systems have entered the open-domain question-answering stage. During this stage, question-answering systems can be further categorized into document-based question-answering systems, question-answer pair-based question-answering systems, and knowledge graph-based question-answering systems, based on the data structures they rely on.

[0003] Document-based question-answering systems, based on user-entered text questions, enable the model to retrieve relevant text segments and extract answers based on its reading comprehension of the document. This approach has the advantage of providing a wide range of knowledge sources, eliminating the need for manual annotation to build a large-scale knowledge base. However, it also places high demands on the model's reading comprehension, retrieval, and answer generation capabilities. In the early days of algorithm development, before pre-trained models were available, the accuracy of question-answering systems was relatively low.

[0004] Question-answering systems based on question-answer pairs mainly focus on frequently asked questions (FAQs) and community question-answering (CQA). FAQ systems require the preparation of standard questions and answers in advance, so the quality of the system's answers is relatively high, but data acquisition is difficult. Patent CN116628146A proposes a method and system for intelligent FAQ answering in the financial field. This method adds similar questions to FAQs. The performance of the retrieval model depends mainly on the quality of the dataset, and the workload of manual annotation is increased. Although the CQA system has a wide range of question-answering data, its answer quality is low due to the non-professional nature of the data source.

[0005] Question-answering systems based on knowledge graphs are highly dependent on the quality and scale of the knowledge graph. They analyze questions, perform queries, and reason within the knowledge graph to ultimately arrive at answers. While their accuracy is relatively high, the cost of labeling knowledge graphs is high, and question-answering systems may not work for unorganized knowledge.

[0006] Furthermore, there are methods for generating data using large language models: This method leverages the understanding capabilities of large language models to generate the required data from unsupervised text. Patent CN117743540A proposes a method and apparatus for collecting datasets for an enterprise intelligent question-answering system. However, it is unclear whether the general large language model used in this method is an offline model. Uploading internal enterprise information to an external general large language model for preprocessing could raise information security concerns.

[0007] And a question-answering system based on a large language model: CN117708309A proposes a retrieval question-answering method, system, device and medium; although this method replaces the answer extraction module of the question-answering system with a general large language model, it does not mention the effective generation of training data sets.

[0008] Therefore, there is still a lot of room for improvement in the current question-answering system, especially the question-answering system for the application field of college admissions consulting. Summary of the Invention

[0009] In order to overcome the shortcomings and deficiencies in the prior art, the purpose of the present invention is to provide a college admissions consulting question-and-answer system based on an offline universal large language model. The system uses an offline universal large language model to replace the answer extraction module of the traditional question-and-answer system, reducing the model training cost while making the question-and-answer system more intelligent. The target document text is first retrieved from the text database using a classification and retrieval model, and then input into the large language model. This can make up for the poor performance of the general large language model in vertical fields, while avoiding the training of the large language model, reducing computing power requirements, and realizing the rapid construction of the question-and-answer system in vertical fields.

[0010] In order to achieve the above-mentioned object, the present invention is implemented through the following technical solutions: a college admissions consultation and answering system based on an offline universal large language model, comprising:

[0011] Input module, used to input the question text of college admissions consultation;

[0012] The classification and retrieval model is used to first classify and predict the input question text to obtain a predicted classification label; based on the predicted classification label, the text database of the corresponding classification is screened from various text databases of universities; then, feature extraction is performed on the question text and each document text in the corresponding classification text database; the features of the question text and each document text are matched for similarity, and the document text with the highest similarity is selected as the output result;

[0013] A combination module is used to organize the question text and the document text output by the classification and retrieval model through the prompt words to form an organized text;

[0014] An offline general-purpose large language model that takes organized text as input and outputs response data for college admissions consultations.

[0015] Preferably, the classification and retrieval model includes a question classification model and a text retrieval model;

[0016] The question classification model is used to classify and predict the input question text to obtain a predicted classification label; based on the predicted classification label, a text database of the corresponding classification is screened out from various text databases of universities;

[0017] The text retrieval model is used to extract features from the question text and each document text in the corresponding classification text database respectively; perform similarity matching on the features of the question text and each document text respectively, and select the document text with high similarity as the output result.

[0018] Preferably, the question classification model includes a mixed granularity feature extraction module 1 and a mixed granularity feature fusion module; the mixed granularity feature extraction module 1 is composed of a plurality of encoding blocks 1 and a splicing layer 1; the mixed granularity feature fusion module is composed of three convolutional layers, three pooling layers, a splicing layer 2 and a fully connected layer 1;

[0019] The question text is input into the mixed granularity feature extraction module 1, encoded by each coding block 1 of the mixed granularity feature extraction module 1, and the output features of each coding block 1 are spliced ​​through the splicing layer 1 to obtain the mixed granularity features of the question text; the mixed granularity feature fusion module extracts features from the mixed granularity features through three convolution layers with convolution kernels of different sizes and processes them through the pooling layer, and then splices the features again through the splicing layer 2 to obtain the final features of the question text, and then classifies it through the fully connected layer 1 to obtain the predicted classification label of the question text.

[0020] Preferably, the text retrieval model includes a second hybrid granularity feature extraction module, a third hybrid granularity feature extraction module, a cross attention module, a feature aggregation layer, and a second fully connected layer; the second hybrid granularity feature extraction module and the third hybrid granularity feature extraction module are respectively composed of a plurality of second encoding blocks and a third splicing layer;

[0021] The question text is input into the hybrid granularity feature extraction module 2, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 2. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature u of the question text; each document text of the corresponding classified text database is input into the hybrid granularity feature extraction module 3, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 3. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature v of each document text; the hybrid granularity features u and v are respectively obtained through the cross attention module to obtain features containing information from both sides Use the feature aggregation layer to obtain aggregated features Finally, after passing through the fully connected layer 2, the similarity score is obtained; the document text with the highest similarity score is selected as the output result.

[0022] Preferably, the question classification model and text retrieval model refer to trained question classification model and text retrieval model;

[0023] The training sample set is obtained in the following way:

[0024] Step X1: collect document texts from various text databases of colleges and universities; collect consultation question records for college admissions and organize them into question text records; and determine classification labels for the document texts and question text records.

[0025] Step X2: Select each classification label as a classification constraint in turn, combine it with the question text record, fill it into prompt 1, and use the offline general language model to generate a question dataset;

[0026] Select unsupervised, unstructured document text and classification labels in turn, fill them into prompt 2, input them into the offline general language model, generate corresponding question data, and obtain the "question-document pair" dataset;

[0027] The question dataset is a training sample set for the question classification model; the “question-document pair” dataset is a training sample set for the text retrieval model.

[0028] Preferably, the question text record is first subjected to data enhancement before step X2 is performed; the data enhancement includes: randomly asking the colleges, majors, and provinces mentioned in the question text record in a template manner; changing the expression of the question text record using a translation method; and using an offline general large language model to perform synonymous expressions on the question text record.

[0029] Preferably, the question classification model is trained using a cross entropy loss function; and the text retrieval model is trained using a triplet loss function.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] 1. This invention replaces the answer extraction module of the traditional question-answering system with an offline universal large language model, reducing model training costs while making the question-answering system more intelligent. It first uses classification and retrieval models to retrieve the target document text from the text database and then inputs it into the large language model. This can overcome the shortcomings of the general large language model in performing poorly in vertical fields, while avoiding the need to train the large language model, reducing computing power requirements, and enabling the rapid construction of question-answering systems in vertical fields.

[0032] 2. The classification and retrieval model of the present invention first classifies the question text, narrowing the scope of the text database to be searched, and then retrieves the target document text from the narrowed text database, effectively improving information retrieval efficiency. The question text and the retrieved document text are then passed to the offline universal large language model. The offline universal large language model's reasoning and comprehension capabilities are used to return the output text as response data, which can reduce the number of answer extraction modules in traditional question-answering systems.

[0033] 3. The present invention utilizes an offline general large language model to solve the data shortage and data security problems faced by question-answering systems when deployed in the field of college admissions consulting. The present invention combines the advantages of the offline general large language model with the classification and retrieval model. When constructing a training data set, the offline general large language model's understanding ability is utilized to quickly generate question data sets and "question-document pair" data sets for question texts and document texts. This provides an effective data set for the training of classification and retrieval models. In addition, the offline operation of the large language model prevents the data and documents provided by vertical fields from being leaked, and thus security and privacy can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the structure of the college admissions consultation question-answering system based on the offline universal large language model of the present invention;

[0035] Figure 2 Schematic diagram of the flow of the question classification model of the present invention;

[0036] Figure 3 Schematic diagram of the structure of the question classification model of the present invention;

[0037] Figure 4 It is a flow chart of the text retrieval model of the present invention;

[0038] Figure 5 It is a structural diagram of the text retrieval model of the present invention;

[0039] Figure 6 It is a flowchart of obtaining a training sample set according to the present invention. DETAILED DESCRIPTION

[0040] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0041] Example

[0042] This embodiment is a college admissions consultation question-answering system based on an offline universal large language model. Figure 1 Shown, including:

[0043] Input module, used to input the question text of college admissions consultation;

[0044] The classification and retrieval model is used to first classify and predict the input question text to obtain a predicted classification label; based on the predicted classification label, the text database of the corresponding classification is screened from various text databases of universities; then, feature extraction is performed on the question text and each document text in the corresponding classification text database; the features of the question text and each document text are matched for similarity, and the document text with the highest similarity is selected as the output result;

[0045] A combination module is used to organize the question text and the document text output by the classification and retrieval model through the prompt words to form an organized text;

[0046] An offline general-purpose large language model that takes organized text as input and outputs response data for college admissions consultations.

[0047] Among them, the third prompt is preferably: Suppose you are an admissions teacher at a certain university and face the following questions from candidates or parents: ( Question text )； The known school situation is as follows: ( Document text output by classification and retrieval models ) Please answer questions from candidates or parents.

[0048] Classification and retrieval models include question classification models and text retrieval models.

[0049] The question classification model is used to classify and predict the input question text and obtain the predicted classification label; according to the predicted classification label, the text database of the corresponding classification is screened out from various text databases of universities, such as Figure 2 shown.

[0050] Question classification model such as Figure 3 As shown, it includes a mixed granularity feature extraction module 1 and a mixed granularity feature fusion module; the mixed granularity feature extraction module 1 is composed of several encoding blocks 1 and a splicing layer 1; the mixed granularity feature fusion module is composed of three convolutional layers, three pooling layers, a splicing layer 2 and a fully connected layer 1.

[0051] The question text is input into the mixed granularity feature extraction module 1, encoded by each coding block 1 of the mixed granularity feature extraction module 1, and the output features of each coding block 1 are spliced ​​through the splicing layer 1 to obtain the mixed granularity features of the question text; the mixed granularity feature fusion module extracts features from the mixed granularity features through three convolutional layers with convolution kernels of different sizes and processes them through the pooling layer, and then splices the features again through the splicing layer 2 to obtain the final features of the question text, and then classifies it through the fully connected layer 1 to obtain the predicted classification label of the question text.

[0052] The text retrieval model is used to extract features from the question text and the corresponding classified document text in the text database; the question text and the features of each document text are matched for similarity, and the document text with high similarity is selected as the output result, such as Figure 4 shown.

[0053] Text retrieval models such as Figure 5As shown, it includes a hybrid granularity feature extraction module 2, a hybrid granularity feature extraction module 3, a cross attention module, a feature aggregation layer and a fully connected layer 2; the hybrid granularity feature extraction module 2 and the hybrid granularity feature extraction module 3 are respectively composed of several encoding blocks 2 and a splicing layer 3;

[0054] The question text is input into the hybrid granularity feature extraction module 2, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 2. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature u of the question text; each document text of the corresponding classified text database is input into the hybrid granularity feature extraction module 3, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 3. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature v of each document text; the hybrid granularity features u and v are respectively obtained through the cross attention module to obtain features containing information from both sides. Use the feature aggregation layer to obtain aggregated features Finally, after passing through the fully connected layer 2, the similarity score is obtained; the document text with the highest similarity score is selected as the output result.

[0055] Question classification model and text retrieval model refer to trained question classification model and text retrieval model;

[0056] This paper proposes a method for constructing a vertical domain question-answering system based on an offline universal large language model. While ensuring data security, it can generate a large amount of training data from unsupervised text in vertical domains and train an intelligent, highly accurate question-answering system, demonstrating its significant practical significance.

[0057] The training sample set is obtained in the following way: Figure 6 As shown:

[0058] Step X1: collect document texts from various text databases of colleges and universities; collect consultation question records for college admissions and organize them into question text records; and determine classification labels for the document texts and question text records.

[0059] In step X2, select each classification label as a classification constraint in turn, combine it with the question text record, fill it into prompt one, and use the offline general large language model to generate a question dataset; prompt one is preferably: Assuming that you are a candidate taking the college entrance examination or their parent, please ask 10 different questions to the university around the topic: (classification label), combined with (question text record).

[0060] Select unsupervised and unstructured document text and classification labels in turn, fill them into prompt 2, input the offline general large language model, generate corresponding question data, and obtain the "question-document pair" data set; prompt 2 is preferably: Assuming that you are a candidate taking the college entrance examination or his / her parent, please ask 10 different questions around the topic: (classification label) that can be answered by the following text: (document text).

[0061] The question dataset is a training sample set for the question classification model; the “question-document pair” dataset is a training sample set for the text retrieval model.

[0062] The question text record is first data enhanced before step X2 is performed; the data enhancement includes: randomly asking the colleges, majors, and provinces mentioned in the question text record using a template; changing the expression of the question text record using a translation method; and using an offline general large language model to perform synonymous expressions on the question text record.

[0063] The question classification model is trained using a cross entropy loss function; the text retrieval model is trained using a triplet loss function.

[0064] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A college admissions consultation question-answering system based on an offline universal large language model, characterized by: include: Input module, used to input the question text of college admissions consultation; The classification and retrieval model is used to first classify and predict the input question text to obtain a predicted classification label; based on the predicted classification label, the text database of the corresponding classification is screened from various text databases of universities; then, feature extraction is performed on the question text and each document text in the corresponding classification text database; the features of the question text and each document text are matched for similarity, and the document text with the highest similarity is selected as the output result; A combination module is used to organize the question text and the document text output by the classification and retrieval model through the prompt words to form an organized text; An offline general-purpose large language model that takes organized text as input and outputs response data for college admissions consultations; The classification and retrieval model includes a question classification model and a text retrieval model; The question classification model includes a mixed granularity feature extraction module 1 and a mixed granularity feature fusion module; the mixed granularity feature extraction module 1 is composed of a plurality of encoding blocks 1 and a splicing layer 1; the mixed granularity feature fusion module is composed of three convolutional layers, three pooling layers, a splicing layer 2 and a fully connected layer 1; The question text is input into the mixed granularity feature extraction module 1, and is encoded by each coding block 1 of the mixed granularity feature extraction module 1. The output features of each coding block 1 are spliced ​​through the splicing layer 1 to obtain the mixed granularity features of the question text; the mixed granularity feature fusion module extracts features from the mixed granularity features through three convolution layers with convolution kernels of different sizes and processes them through the pooling layer, and then splices the features again through the splicing layer 2 to obtain the final features of the question text, and then classifies them through the fully connected layer 1 to obtain the predicted classification label of the question text; The text retrieval model includes a second hybrid granularity feature extraction module, a third hybrid granularity feature extraction module, a cross attention module, a feature aggregation layer, and a second fully connected layer; the second hybrid granularity feature extraction module and the third hybrid granularity feature extraction module are respectively composed of a plurality of second encoding blocks and a third splicing layer; The question text is input into the hybrid granularity feature extraction module 2, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 2. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature u of the question text; each document text of the corresponding classified text database is input into the hybrid granularity feature extraction module 3, and is encoded by each coding block 2 of the hybrid granularity feature extraction module 3. The output features of each coding block 2 are spliced ​​through the splicing layer 2 to obtain the hybrid granularity feature v of each document text; the hybrid granularity features u and v are respectively obtained through the cross attention module to obtain features containing information from both sides Use the feature aggregation layer to obtain aggregated features Finally, after passing through the fully connected layer 2, the similarity score is obtained; The document text with the highest similarity score is selected as the output result.

2. The college admissions consultation question-answering system based on an offline universal large language model according to claim 1 is characterized by: The question classification model is used to classify and predict the input question text to obtain a predicted classification label; based on the predicted classification label, a text database of the corresponding classification is screened out from various text databases of universities; The text retrieval model is used to extract features from the question text and each document text in the corresponding classification text database respectively; perform similarity matching on the features of the question text and each document text respectively, and select the document text with high similarity as the output result.

3. The college admissions consultation question-answering system based on an offline universal large language model according to claim 2 is characterized by: The question classification model and text retrieval model refer to trained question classification models and text retrieval models; The training sample set is obtained in the following way: Step X1: collect document texts from various text databases of colleges and universities; collect consultation question records for college admissions and organize them into question text records; and determine classification labels for the document texts and question text records. Step X2: Select each classification label as a classification constraint in turn, combine it with the question text record, fill it into prompt 1, and use the offline general language model to generate a question dataset; Select the document text and category label in sequence, fill them into prompt 2, input the offline general language model, generate the corresponding question data, and obtain the "question-document pair" dataset; The question dataset is a training sample set for the question classification model; the "question-document pair" dataset is a training sample set for the text retrieval model.

4. The college admissions consultation question-answering system based on an offline universal large language model according to claim 3 is characterized by: The question text record is first subjected to data enhancement before step X2 is performed; the data enhancement includes: randomly adding the colleges, majors, and provinces mentioned in the question text record in a template manner; changing the expression of the question text record using a translation method; and using an offline general large language model to perform synonymous expressions on the question text record.

5. The college admissions consultation question-answering system based on an offline universal large language model according to claim 3 is characterized by: The question classification model is trained using a cross entropy loss function; the text retrieval model is trained using a triplet loss function.

Images