Construction method of river crab knowledge question-answering system Crab KB

Abstract

The invention relates to a method for constructing a river crab knowledge question-answering system Crab KB, which comprises the following steps of: acquiring data to obtain an original text, generating a question-answering pair by utilizing the acquired original text, and extracting document blocks from the acquired original text; performing quality evaluation and screening on the original text and document blocks; and continuing to perform pre-training and supervised fine tuning on the evaluation model, and continuously optimizing and updating the evaluation model to obtain the river crab knowledge question-answering system Crab KB. The beneficial effects of the invention are that a river crab knowledge question-answering system Crab KB which meets the requirements of daily learning, maintenance, management, brand maintenance and the like of river crab farmers is constructed, and the training and learning costs of the river crab farmers can be reduced; through more refined model training and using strategies, the problems that an existing river crab knowledge question-answering system lacks deep understanding of knowledge in a specific field, a local knowledge base is difficult to integrate effectively, the knowledge base is difficult to maintain and the like can be solved.

Description

Technical Field

[0001] The invention belongs to the field of information technology and agricultural breeding technology, and in particular relates to a method for constructing a river crab knowledge question-answering system CrabKB. Background Art

[0002] Existing intelligent question-answering systems built on large models usually rely on pre-trained language models (such as the models in the Hugging Face Transformers library) to understand and generate natural language text to answer questions. Although these intelligent question-answering systems built on large models have extensive knowledge, because the knowledge of the pre-trained language models is mainly obtained through training on a large amount of general text, rather than being optimized for a knowledge base specifically for a certain field, they often lack the depth and accuracy of specific field or localized knowledge, which greatly limits their effectiveness in practical applications.

[0003] When it comes to combining with local knowledge bases, traditional fine-tuning methods may not be enough for the model to fully absorb and utilize this expertise, making it difficult to effectively integrate local knowledge bases. This also results in the model not being able to provide sufficiently accurate answers when processing queries involving specific domain knowledge.

[0004] Existing knowledge question answering systems may face challenges in maintaining and updating the knowledge base. As domain knowledge continues to develop and change, the system needs to regularly update the knowledge base to ensure the accuracy and timeliness of the information. However, this process is often manual and time-consuming, limiting the flexibility and responsiveness of the system.

[0005] In summary, the existing Hexie knowledge question-answering system lacks a deep understanding of knowledge in specific fields, has difficulty in effectively integrating local knowledge bases, and has difficulty maintaining the knowledge base. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for constructing a crab knowledge question answering system CrabKB.

[0007] The method for constructing the CrabKB knowledge question-answering system includes the following steps:

[0008] Step 1: Collect data to obtain original text, use the collected original text to generate question-answer pairs, and extract document blocks from the collected original text;

[0009] Step 2: Input the original text and document blocks into the evaluation model, and the evaluation model performs quality evaluation and screening on the original text and document blocks;

[0010] Step 3: Continue pre-training and supervised fine-tuning of the evaluation model, and continuously optimize and update the evaluation model to adapt to new data and needs, and obtain the CrabKB knowledge question-answering system.

[0011] Preferably, the data collection in step 1 to obtain the original text includes: using crawler technology to capture the original text related to river crab farming from the Internet, which solves the problem of difficulty in updating the knowledge base in the traditional question-and-answer system and ensures the timeliness and richness of the information.

[0012] Preferably, the original texts include book texts, journal texts, patent texts and news texts.

[0013] Preferably, the evaluation model in step 2 is the ChatGLM model.

[0014] Preferably, step 2 specifically comprises the following steps:

[0015] Step 2.1, input the original text and document blocks into the ChatGLM model. The ChatGLM model uses the timestamps in the results returned by the search engine during crawling to determine the freshness of the original text and the documents to which the document blocks belong;

[0016] Step 2.2, determining the authority of the documents to which the original text and the document blocks belong, and dividing the sources of the original text and the document blocks according to their authority;

[0017] Step 2.3: Use qualitative indicators to train the ChatGLM model, and use the trained ChatGLM model to perform hierarchical processing on the original text and document blocks.

[0018] As a preference: in step 2.1, the freshness of the document to which the original text belongs includes the publication date of the book, the publication date of the journal, the application date of the patent and the release date of the news; in step 2.2, the authority of the document to which the original text belongs is judged based on the official attributes of the source of the original text and the author information, and the source of the original text is divided into authority categories; in step 2.3, the qualitative indicators include professionalism and popularity.

[0019] As a preferred method, step 3 is to continue pre-training and supervised fine-tuning the evaluation model based on the open source framework LangChain and the library Faiss (the combination of LangChain and Faiss provides the system with powerful asynchronous processing capabilities and efficient vector retrieval functions, making the question-answering process smoother and more responsive), and continuously optimize and update the evaluation model to adapt to new data and needs; the detailed steps are:

[0020] Step 3.1: Based on the open source framework LangChain and library Faiss, continue to pre-train the evaluation model;

[0021] Step 3.2: Optimize the model parameters within the probability distribution of each sequence according to the pre-trained ChatGLM model in the following formula (1) by minimizing the negative log-likelihood (NLL):

[0022]

[0023] In the above formula, θ represents the set of all parameters of the ChatGLM model, O(θ) represents the loss function of the ChatGLM model; D represents the training data set, x represents the sample in the training data set, ∑ x∈D represents the sum of all samples x in the data set D; L represents the length of the sequence x, logp θ (x) represents the negative log-likelihood of a single sample x, p θ (x) is the probability of sample x under the model parameter set θ; It means to sum each word or character in the sample x until the Lth element of the sample, where L is the length of the sample; logp θ (x i ∣x 1:i―1 ) represents the negative logarithm of the conditional probability of the i-th sample when the first i-1 samples are known, x i is the i-th sample, and x 1:i―1 is the sequence of the first i-1 samples;

[0024] Step 3.3: The input data of the ChatGLM model is in the format of question-answer pairs (instructions + answers). The instructions of the ChatGLM model will predict the next sequence. The ChatGLM model processed in step 3.2 is fine-tuned using the supervised fine-tuning technique SFT. After the supervised fine-tuning technique SFT is used, the output of the ChatGLM model is usually an inactivated output (logits tensor with a dimension of (1, T, vocab_size), where T is the length of the sequence). The difference between the inactivated output of the ChatGLM model corresponding to the aligned answer data and the next sequence prediction is reduced:

[0025]

[0026] In the above formula, N is the size of the vocabulary vocab_size (for a common 7B model, this size vocab_size usually ranges from 120,000 to 160,000, depending on the field); i represents the category of the data; yi is the one-hot encoding of the true label, if the sample of the input data belongs to category i, then yi = 1, otherwise yi = 0; pi is the probability that the ChatGLM model predicts that the sample belongs to category i;

[0027] Step 3.4, introduce the mask vector and ignore the loss corresponding to the padding value to calculate the loss function of the ChatGLM model processed by step 3.3; the calculation result of the loss function is used to guide the adjustment of the parameters of the ChatGLM model to obtain the CrabKB knowledge question answering system.

[0028] Preferably, the number of parameters of the ChatGLM model is 7b (7 billion) or 13b (13 billion).

[0029] The beneficial effects of the present invention are:

[0030] The method of the present invention constructs a crab knowledge question-and-answer system CrabKB that meets the daily learning, maintenance, management, and brand maintenance needs of crab farmers, and can reduce the training and learning costs of crab farmers. The present invention can solve the problems of the existing crab knowledge question-and-answer system lacking a deep understanding of knowledge in specific fields, being difficult to effectively integrate local knowledge bases, and having difficulty maintaining knowledge bases through more refined model training and usage strategies:

[0031] 1) In view of the problem that the existing technology lacks in-depth understanding of specific field knowledge, the present invention continuously collects river crab-related industry documents by directional combing of industry knowledge types, and organizes the documents into text blocks, question-answer pairs, and original files. This not only ensures the continuous updating of the knowledge base, but also enables the system to better understand and master the specific knowledge and application scenarios of the river crab industry;

[0032] 2) In order to solve the problem of difficulty in effectively integrating local knowledge bases, the present invention introduces the ChatGLM model of industry real-time, industry professionalism, industry authority, etc. to classify and label the collected data. This can not only effectively integrate and utilize the existing local knowledge base, but also further improve the quality and availability of the knowledge base;

[0033] 3) In view of the difficulty in maintaining and updating the knowledge base in the prior art, the present invention can effectively manage and update the data in the knowledge base, further reducing the difficulty of knowledge base maintenance.

[0034] The crab knowledge question-and-answer system CrabKB used in the present invention aims to create a system that can continuously learn, self-optimize and provide high-quality question-and-answer services for the crab industry by integrating automated crawler technology, intelligent evaluation and screening mechanism and efficient asynchronous question-and-answer processing framework; the architecture of the system not only improves the performance of large models, but also enhances its ability to combine with industry knowledge to provide users with more accurate and professional answers. The method of the present invention improves the performance and adaptability of the crab knowledge question-and-answer system in crab farming technology.

[0035] Based on the open source framework LangChain and library Faiss, the present invention continues to pre-train and supervise fine-tune the ChatGLM model; the combination of LangChain and Faiss provides the system with powerful asynchronous processing capabilities and efficient vector retrieval functions, making the question-answering process smoother and more responsive; Faiss provides a variety of index types, supports fast retrieval of high-dimensional vector data, and can process data sets that cannot be fully placed in memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flowchart of the method for constructing the crab knowledge question-answering system CrabKB. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with embodiments. The description of the following embodiments is only used to help understand the present invention. It should be noted that for ordinary persons in the art, without departing from the principle of the present invention, the present invention can also be modified in some ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

[0038] Example 1

[0039] like Figure 1 As shown, a method for constructing a crab knowledge question answering system CrabKB includes the following steps:

[0040] Step 1: Use crawler technology to crawl original texts (book texts, journal texts, patent texts and news texts) related to river crab farming from the Internet, solve the problem of difficulty in updating the knowledge base in the traditional question-and-answer system, ensure the timeliness and richness of information, use the collected original texts to generate question-and-answer pairs, and extract document blocks from the collected original texts; construct a benchmark question-and-answer data set CrabBench in the field of river crabs; the CrabBench data set is collected from the river crab farming professional qualification examination, aquaculture physician professional certificate examination, agricultural aquaculture department management certification examination, and practicing veterinary qualification certificate (aquatic biology) examination on the Internet. 800 objective questions are collected, and experts are invited to review and determine their rationality and effectiveness; from the assessment aspect, this embodiment divides the data set CrabBench into 6 categories of questions, including aquaculture laws and regulations, seedling selection, feed management, disease prevention and control, breeding management, and breeding site management.

[0041] Step 2: Input the original text and document blocks into the ChatGLM model (the number of parameters of the ChatGLM model is 7b (7 billion)), and the ChatGLM model performs quality assessment and screening on the original text and document blocks;

[0042] Step 2.1, input the original text and document blocks into the ChatGLM model. The ChatGLM model uses the timestamps in the results returned by the search engine during crawling to determine the freshness of the original text and the documents to which the document blocks belong; freshness includes the publication date of books, the publication date of journals, the application date of patents, and the release date of news;

[0043] Step 2.2: Determine the authority of the document to which the original text belongs based on the official attributes of the source and the author information of the original text, and divide the source of the original text into authoritative categories;

[0044] For example, from the perspective of news sources: According to Table 1 below, source information can be divided and then authoritative division can be achieved.

[0045] Table 1 Authority of news

[0046]

[0047]

[0048] In addition, there are relatively clear classification standards for books, periodicals, patents, etc.

[0049] 1) For books:

[0050] Level 1: The number of various types of book publishing units accounts for about 20% of the total number of commercial book publishing units in the country. The 100 book publishing units that are rated as level 1 for the first time will be awarded the honorary title of "Top 100 Book Publishing Units in the Country" and a preferential policy will be implemented in the allocation of publishing resources.

[0051] Level 2: The number of various types of book publishing units accounts for about 30% of the total number of commercial book publishing units in the country.

[0052] Level 3: The number of various types of book publishing units accounts for about 40% of the total number of commercial book publishing units in the country.

[0053] Level 4: The proportion of all types of book publishing units does not exceed 10% of the total number of commercial book publishing units in the country. For book publishing units rated as Level 4, their supervisors and sponsors are required to strengthen management, promote them to improve various systems, improve business management, and enhance their comprehensive strength.

[0054] 2) For journals: mainly domestic journals and SCI journal papers. The authority of domestic journals mainly follows indicators such as division and number of citations, such as EI, CSCD, Chinese Core Journal, etc., while SCI journal papers are mainly divided according to the Chinese Academy of Sciences journal division or JCR division, such as Chinese Academy of Sciences Divisions 1 to 4.

[0055] Step 2.3, use qualitative indicators to train the ChatGLM model, and use the trained ChatGLM model to perform hierarchical processing on the original text and document blocks; qualitative indicators include professionalism and popularity;

[0056] For example, ordinary news and Chinese journal papers or English SCI journal papers are classified at a lower level. Professional books, high-level SCI papers or journals with relatively narrow fields are classified at a higher level. The specific classification is shown in Table 2 below:

[0057] Table 2 Professional classification table of book sources

[0058]

[0059]

[0060] It can be seen that hierarchical processing of original text and document blocks helps improve the system's ability to understand the knowledge of a specific industry (such as the crab industry), ensuring that the information provided is both real-time and authoritative and professional;

[0061] Step 3: Based on the open source framework LangChain and the library Faiss (the combination of LangChain and Faiss provides the system with powerful asynchronous processing capabilities and efficient vector retrieval functions, making the question-answering process smoother and more responsive), continue to pre-train and supervise the fine-tuning of the ChatGLM model, and continuously update the ChatGLM model to adapt to new data and needs, and obtain the CrabKB knowledge question-answering system:

[0062] Step 3.1: Based on the open source framework LangChain and library Faiss, continue to pre-train the evaluation model;

[0063] Step 3.2: Optimize the model parameters within the probability distribution of each sequence according to the pre-trained ChatGLM model in the following formula (1) by minimizing the negative log-likelihood (NLL):

[0064]

[0065] In the above formula, θ represents the set of all parameters of the ChatGLM model, O(θ) represents the loss function of the ChatGLM model; D represents the training data set, x represents the sample in the training data set, ∑ x∈D represents the sum of all samples x in the data set D; L represents the length of the sequence x, logp θ (x) represents the negative log-likelihood of a single sample x, p θ (x) is the probability of sample x under the model parameter set θ; It means to sum each word or character in the sample x until the Lth element of the sample, where l represents the length of the sample; logp θ (x i ∣x 1:i―1 ) represents the negative logarithm of the conditional probability of the i-th sample when the first i-1 samples are known, x i is the i-th sample, and x 1:i―1 is the sequence of the first i-1 samples;

[0066] Step 3.3: The input data of the ChatGLM model is in the format of question-answer pairs (instructions + answers). The instructions of the ChatGLM model will predict the next sequence. The ChatGLM model processed in step 3.2 is fine-tuned using the supervised fine-tuning technique SFT. After the supervised fine-tuning technique SFT is used, the output of the ChatGLM model is usually an inactivated output (logits tensor with a dimension of (1, T, vocab_size), where T is the length of the sequence). The difference between the inactivated output of the ChatGLM model corresponding to the aligned answer data and the next sequence prediction is reduced:

[0067]

[0068] In the above formula, N is the size of the vocabulary vocab_size (for a common 7B model, this size vocab_size usually ranges from 120,000 to 160,000, depending on the field); i represents the category of the data; yi is the one-hot encoding of the true label, if the sample of the input data belongs to category i, then yi = 1, otherwise yi = 0; pi is the probability that the ChatGLM model predicts that the sample belongs to category i;

[0069] Step 3.4, introduce the mask vector and ignore the loss corresponding to the padding value to calculate the loss function of the ChatGLM model processed by step 3.3; the calculation result of the loss function is used to guide the adjustment of the parameters of the ChatGLM model to obtain the CrabKB knowledge question-answering system (also known as CrabChat).

[0070] The following Table 3 is a quantitative performance comparison table of the general artificial intelligence large model GPT3.5, the open source large model LLaMA-3-8B, Alpaca-7B and CrabChat designed in this embodiment.

[0071] Table 3 Performance comparison of CrabChat and general large models in the crab domain (unit: accuracy)

[0072]

[0073] The experimental results show that in the field of river crab farming, the river crab knowledge question and answer system CrabKB (also known as CrabChat) adopted in this embodiment has surpassed the mainstream chatGPT and meta open source LLaMA-3 and other open source general field large models in objective question and answer tasks in the field of river crab farming, verifying the effectiveness of the method of the present invention.

[0074] Example 2

[0075] On the basis of Example 1, another method for constructing CrabKB, a crab knowledge question answering system, changes the number of parameters of the ChatGLM model to 13b (13 billion). The rest of the method is the same as that in Example 1.

Claims

1. A method for constructing a crab knowledge question-answering system CrabKB, characterized in that: The following steps are involved: Step 1: Collect data to obtain original text, use the collected original text to generate question-answer pairs, and extract document blocks from the collected original text; Step 2: Input the original text and document blocks into the evaluation model, and the evaluation model performs quality evaluation and screening on the original text and document blocks; Step 3: Continue pre-training and supervised fine-tuning the evaluation model, continuously optimize and update the evaluation model, and obtain the CrabKB knowledge question answering system.

2. The method for constructing the crab knowledge question-answering system CrabKB according to claim 1 is characterized in that: The data collection in step 1 to obtain the original text includes: using crawler technology to crawl the original text related to river crab farming from the Internet.

3. The method for constructing the crab knowledge question-answering system CrabKB according to claim 2 is characterized by: Original texts include book texts, journal texts, patent texts, and news texts.

4. The method for constructing the crab knowledge question-answering system CrabKB according to claim 1 is characterized by: The evaluation model in step 2 is the ChatGLM model.

5. The method for constructing the crab knowledge question answering system CrabKB according to claim 4 is characterized in that: Step 2 specifically includes the following steps: Step 2.1, input the original text and document blocks into the ChatGLM model. The ChatGLM model uses the timestamps in the results returned by the search engine during crawling to determine the freshness of the original text and the documents to which the document blocks belong; Step 2.2, determining the authority of the documents to which the original text and the document blocks belong, and dividing the sources of the original text and the document blocks according to their authority; Step 2.3: Use qualitative indicators to train the ChatGLM model, and use the trained ChatGLM model to perform hierarchical processing on the original text and document blocks.

6. The method for constructing the crab knowledge question answering system CrabKB according to claim 5 is characterized by: In step 2.1, the freshness of the document to which the original text belongs includes the publication date of the book, the publication date of the journal, the application date of the patent, and the release date of the news; in step 2.2, the authority of the document to which the original text belongs is judged based on the official attributes of the source of the original text and the author information, and the authority of the source of the original text is divided; in step 2.3, the qualitative indicators include professionalism and popularity.

7. The method for constructing the crab knowledge question-answering system CrabKB according to claim 5 is characterized in that: Step 3 is to continue pre-training and supervised fine-tuning the evaluation model based on the open source framework LangChain and library Faiss, and continuously optimize and update the evaluation model; the detailed steps are: Step 3.1: Based on the open source framework LangChain and library Faiss, continue to pre-train the evaluation model; Step 3.2: Optimize the model parameters within the probability distribution of each sequence according to the pre-trained ChatGLM model in the following formula (1) by minimizing the negative log-likelihood: In the above formula, θ represents the set of all parameters of the ChatGLM model, O(θ) represents the loss function of the ChatGLM model; D represents the training data set, x represents the sample in the training data set, ∑ x∈D represents the sum of all samples x in the data set D; L represents the length of the sequence x, logp θ (x) represents the negative log-likelihood of a single sample x, p θ (x) is the probability of sample x under the model parameter set θ; It means to sum each word or character in the sample x until the Lth element of the sample, where l represents the length of the sample; logp θ (x i ∣x 1:i―1 ) represents the negative logarithm of the conditional probability of the i-th sample when the first i-1 samples are known, x i is the i-th sample, and x 1:i―1 is the sequence of the first i-1 samples; Step 3.3, supervised fine-tuning technique SFT is used to fine-tune the ChatGLM model processed in step 3.2 to reduce the difference between the inactivated output of the ChatGLM model corresponding to the aligned answer data and the prediction of the next sequence: In the above formula, N is the size of the vocabulary vocab_size; i represents the category of the data; yi is the unique hot encoding of the true label, if the sample of the input data belongs to category i, then yi = 1, otherwise yi = 0; pi is the probability that the ChatGLM model predicts that the sample belongs to category i; Step 3.4, introduce the mask vector and ignore the loss corresponding to the filling value to calculate the loss function of the ChatGLM model processed by step 3.3; the calculation result of the loss function is used to guide the adjustment of the parameters of the ChatGLM model to obtain the CrabKB knowledge question answering system.

8. The method for constructing the crab knowledge question answering system CrabKB according to claim 7 is characterized by: The number of parameters for the ChatGLM model is 7b or 13b.

Images