Question and answer model training method and device, question and answer method and device and terminal equipment

Abstract

The invention is suitable for the technical field of computers, and provides a question and answer model training method, a question and answer method, a question and answer device and a terminal device, the question and answer model training method comprises: obtaining a first question and answer data set; the first question and answer data set comprises more than two specific enhanced samples, and the specific enhanced samples comprise specific questions and answers with reference documents corresponding to the specific questions; taking a specific question in the specific enhanced sample and the corresponding answer with the reference document as a model input sample and a model output sample respectively, and performing enhanced training on the basic question and answer model to obtain a target question and answer model; the basic question-answer model is obtained based on training of a second question-answer data set, and the second question-answer data set comprises a plurality of questions and answers corresponding to the questions. Through the method, the target question and answer model has the capability of generating the answer information with the reference document, and more accurate and authoritative service support can be provided in subsequent specific work.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a question-answering model training method, a question-answering method, an apparatus, and a terminal device. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the outstanding strength of large-scale pre-trained language models in diversified tasks, their application in specific work is becoming more and more extensive. In specific work, the use of large-scale pre-trained language models can realize knowledge question answering, that is, through intelligent analysis of consulting sentences or actual scene texts input into large-scale pre-trained language models, large-scale pre-trained language models understand and analyze the input content and generate corresponding answers or suggestions.

[0003] However, current large-scale pre-trained language models do not provide reference documents corresponding to the answers when outputting them. Due to the lack of support from reference documents, the credibility and authority of the output answers are reduced. In situations involving legal liability, it may also lead to unclear attribution of legal liability and failure to provide clear evidentiary support for legal liability. Summary of the invention

[0004] The embodiments of the present application provide a question-answering model training method, a question-answering method, an apparatus and a terminal device, which can solve the problem that the current large-scale pre-trained language model does not provide reference documents corresponding to the answers when outputting answers, resulting in low credibility and authority and unclear legal liability.

[0005] In a first aspect, an embodiment of the present application provides a question-answering model training method, comprising:

[0006] Obtain a first question-answering dataset; the first question-answering dataset includes more than two specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question;

[0007] The specific question in the specific enhanced sample and the corresponding answer with the reference document are used as the model input sample and the model output sample respectively, and the basic question and answer model is enhanced trained to obtain the target question and answer model; the basic question and answer model is trained based on the second question and answer data set, and the second question and answer data set includes multiple questions and answers corresponding to each of the questions.

[0008] In a possible implementation manner of the first aspect, before obtaining the first question-answer data set, the method further includes:

[0009] For the input specific question, searching a preset knowledge base for at least one document related to the specific question;

[0010] Inputting the specific question, the searched documents related to the specific question and the preset guidance text into a preset language model to obtain an answer with referenced documents corresponding to the specific question, wherein the guidance text is used to guide the result output mode of the preset language model;

[0011] The specific enhanced sample is generated based on the specific question and the corresponding answer with the reference document.

[0012] In a possible implementation manner of the first aspect, after obtaining the target question-answering model, the method further includes:

[0013] Obtain a preference data set; the preference data set includes positive sample data and negative sample data; the positive sample data includes a specific question sample and a corresponding reference answer, and the reference answer has a reference document; the negative sample data includes the specific question sample and an answer output by the target question-answering model based on the specific question sample;

[0014] Based on the preference data set, preference alignment optimization training is performed on the target question answering model to obtain an optimized target question answering model.

[0015] In a possible implementation manner of the first aspect, performing preference alignment optimization training on the target question answering model based on the preference dataset includes:

[0016] Inputting a specific question sample in the positive sample data into the current target question-answering model, obtaining a prediction result corresponding to the positive sample data, and calculating a first loss between the prediction result corresponding to the positive sample data and the reference answer;

[0017] Inputting a specific question sample in the negative sample data into the current target question-answering model, obtaining a prediction result corresponding to the negative sample data, and calculating a second loss between the prediction result corresponding to the negative sample data and a corresponding answer in the negative sample data;

[0018] Performing a weighted summation on the first loss and the second loss to obtain a target loss, wherein the weight of the second loss is less than the weight of the first loss;

[0019] Iteratively update the model parameters of the current target question-answering model according to the target loss until the target question-answering model meets a preset first training stop condition.

[0020] In a possible implementation manner of the first aspect, obtaining the first question and answer dataset includes:

[0021] Based on the number of questions included in the second question and answer dataset, a corresponding number of specific enhanced samples are obtained in a preset ratio to obtain the first question and answer dataset, wherein the number of specific enhanced samples included in the first question and answer dataset is less than the number of questions included in the second question and answer dataset.

[0022] In a possible implementation manner of the first aspect, using the specific question in the specific enhanced sample and the corresponding answer with the reference document as a model input sample and a model output sample respectively, and performing enhanced training on the basic question-answering model, includes:

[0023] Inputting the specific question in the specific enhanced sample into the basic question-answering model to obtain an initial predicted answer output by the basic question-answering model;

[0024] Calculating the loss between the initial prediction answer and the corresponding model output sample;

[0025] The model parameters of the basic question-answering model are iteratively updated according to the loss between the initial predicted answer and the corresponding model output sample until the basic question-answering model meets a preset second training stop condition.

[0026] In a second aspect, an embodiment of the present application provides a question-answering method, including:

[0027] Get the target question;

[0028] The target question is input into a target question-answering model to obtain an answer with a reference document corresponding to the target question, wherein the target question-answering model is trained using the question-answering model training method as described in any one of the first aspects above.

[0029] In a third aspect, an embodiment of the present application provides a question-answering model training device, comprising:

[0030] A first acquisition module is used to acquire a first question-answering dataset; the first question-answering dataset includes more than two specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question;

[0031] An enhanced training module is used to use the specific question in the specific enhanced sample and the corresponding answer with the reference document as a model input sample and a model output sample, respectively, to perform enhanced training on the basic question and answer model to obtain a target question and answer model; the basic question and answer model is trained based on a second question and answer data set, and the second question and answer data set includes multiple questions and answers corresponding to each of the questions.

[0032] In a fourth aspect, an embodiment of the present application provides a question-answering device, including:

[0033] The second acquisition module is used to acquire the target question;

[0034] A generation module is used to input the target question into a target question-answering model to obtain an answer with a reference document corresponding to the target question, wherein the target question-answering model is trained using the question-answering model training method described in any one of the first aspects above.

[0035] In a fifth aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the question-answering model training method as described in any one of the first aspects above is implemented; or, when the processor executes the computer program, the question-answering method as described in the second aspect above is implemented.

[0036] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the question-answering model training method as described in any one of the first aspects above; or, when the computer program is executed by a processor, the computer program implements the question-answering method as described in the second aspect above.

[0037] In the seventh aspect, an embodiment of the present application provides a computer program product, which, when the computer program product is running on a terminal device, enables the terminal device to execute the question-answering model training method described in any one of the first aspects above; or, when the computer program product is running on a terminal device, enables the terminal device to execute the question-answering method described in the second aspect above.

[0038] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0039] The question-answering model training method provided in the first aspect of the embodiment of the present application, first, by obtaining a specific enhanced sample containing a specific question and an answer with a reference document corresponding to the specific question in a specific field, reliable source information can be attached to the specific question, so that the answer corresponding to the specific question has high accuracy and authority; then, by introducing the specific question in the specific enhanced sample as a model input sample, and taking the answer with a reference document corresponding to the specific question as a model output sample, the basic question-answering model is enhanced and trained, so that the basic question-answering model (i.e., the target question-answering model) after enhanced training can understand the user's question more accurately, and quickly generate an answer that meets specific requirements and has authority, that is, the target question-answering model has the ability to generate answer information with reference documents. The question-answering method provided in the second aspect of the embodiment of the present application, when a specific person is dealing with a target problem, can use the target question-answering model to generate an answer with a reference document corresponding to the target problem, and make a more wise and accurate decision based on the answer with a reference document generated by the target question-answering model, so as to improve the credibility and persuasiveness of the decision made by the specific person when dealing with the target problem.

[0040] It is understood that the beneficial effects of the third aspect, the fifth aspect to the seventh aspect can refer to the relevant description in the first aspect, and will not be repeated here. The beneficial effects of the fourth aspect, the fifth aspect to the seventh aspect can refer to the relevant description in the second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 It is a flowchart of a question-answering model training method provided in one embodiment of the present application;

[0043] Figure 2 It is a flowchart of another question-answering model training method provided in one embodiment of the present application;

[0044] Figure 3 It is a flowchart of a specific example of generating a specific enhanced sample provided by an embodiment of the present application;

[0045] Figure 4 is a flowchart of the sub-steps of step S105 provided in an embodiment of the present application;

[0046] Figure 5It is a flowchart of another question-answering model training method provided in one embodiment of the present application;

[0047] Figure 6 is a flowchart of the sub-steps of step S107 provided in an embodiment of the present application;

[0048] Figure 7 It is an overall schematic diagram of the ORPO optimization method provided in one embodiment of the present application;

[0049] Figure 8 It is a flowchart of a question-and-answer method provided in one embodiment of the present application;

[0050] Fig. 9 It is a structural diagram of a question-answering model training device provided in one embodiment of the present application;

[0051] Fig.10 is a structural diagram of a specific question-and-answer device provided in one embodiment of the present application;

[0052] Fig.11 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0053] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0054] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0055] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0056] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0057] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0059] With the rapid development of artificial intelligence technology, especially the outstanding strength of large-scale pre-trained language models in diversified tasks, their application in specific work is becoming more and more extensive. In specific work, the use of large-scale pre-trained language models can realize knowledge question answering, that is, through intelligent analysis of consulting sentences or actual scene texts input into large-scale pre-trained language models, large-scale pre-trained language models understand and analyze the input content and generate corresponding answers or suggestions.

[0060] However, current large-scale pre-trained language models do not provide reference documents corresponding to the answers when outputting them. Due to the lack of support from reference documents, the credibility and authority of the output answers are reduced. In situations involving legal liability, it may also lead to unclear attribution of legal liability and failure to provide clear evidentiary support for legal liability.

[0061] In order to solve the above technical problems, an embodiment of the present application provides a question-answering model training method. First, by obtaining a specific enhanced sample in a specific field that contains a specific question and an answer with a reference document corresponding to the specific question, reliable source information can be attached to the specific question, so that the answer corresponding to the specific question has higher accuracy and authority; then, by introducing the specific question in the specific enhanced sample as a model input sample, and taking the answer with a reference document corresponding to the specific question as a model output sample, the basic question-answering model is enhanced trained, so that the basic question-answering model after enhanced training (i.e., the target question-answering model) can understand the user's questions more accurately and quickly generate answers that meet specific requirements and are authoritative, that is, the target question-answering model has the ability to generate answer information with reference documents.

[0062] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0063] Figure 1 FIG. 1 is a flow chart of a question-answering model training method provided in an embodiment of the present application. Figure 1 As shown, the question-answering model training method includes step S104 and step S105.

[0064] Step S104: Obtain a first question-answering dataset; the first question-answering dataset includes two or more specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question.

[0065] In order to quickly obtain specific enhanced samples, first, a large number of specific question-answer data pairs are collected. In specific scenarios, such as legal consultation, case investigation, and public safety, specific question-answer data generated in specific work can be collected through a variety of different sources. Among them, the various different sources include open source legal data, legal and regulatory documents, legal question-answer books, etc. After obtaining a large amount of specific question-answer data, a large number of specific question-answer data pairs can be obtained by analyzing and processing the specific question-answer data. Among them, each specific question-answer data pair usually includes three main parts: a specific question, a document related to the specific question, and an answer corresponding to the specific question. Among them, the specific question is a specific question related to a specific work; the document related to the specific question is a reference material that supports or explains the answer to the specific question, which has high authority and accuracy; the answer corresponding to the specific question is a specific answer given to the specific question.

[0066] Specifically, for specific question-and-answer data in open source legal data, there are many data pairs that already have a direct "question-and-answer" structure, that is, the specific question-and-answer data already exists in the format of specific question-and-answer data pairs. By extracting these specific question-and-answer data pairs, the specific question-and-answer data pairs contained in the open source legal data can be directly obtained.

[0067] For specific question-and-answer data in legal documents, these legal documents often do not have a direct "question-and-answer" structure, so it is necessary to extract the specific question-and-answer data pairs contained in the legal documents. The specific method of extracting specific question-and-answer data pairs from legal documents is: first, split them according to paragraphs, chapters or specific keywords to identify the parts related to the specific; then, use regular expressions to quickly locate text blocks containing specific keywords or patterns; finally, use other natural language processing technologies (such as named entity recognition, syntactic analysis, relation extraction, etc.) to extract specific information related to the specific, so as to obtain the specific question-and-answer data pairs contained in the legal documents.

[0068] Specific question-and-answer data in legal question-and-answer books are often stored in epub and other format files. In order to obtain specific question-and-answer data pairs in legal question-and-answer books, first, use a dedicated epub document parsing tool to extract text content; then, use HTML and XML parsing libraries such as BeautifulSoup to further parse and structure the extracted text content; finally, analyze the document structure (such as titles, paragraphs, lists, etc.) to identify and extract specific question-and-answer data pairs in legal question-and-answer books.

[0069] After collecting a large number of specific question-answer data pairs, a preset knowledge base, i.e., a specific knowledge base, can be constructed using a large number of specific question-answer data pairs, so that specific personnel can query and quote content related to specific work at any time. For example, in a specific work, a specific person can quickly query relevant specific knowledge, actual cases, laws and regulations, etc. in a specific knowledge base by using keywords, question types, etc., so as to make correct decisions when handling cases and providing legal advice. When handling cases or providing legal advice, specific personnel need to quote relevant laws and regulations or actual cases to make decisions. Through the reference function of the specific knowledge base, specific personnel can easily find and quote the laws and regulations and other content they have found, and use them as important references and reference sources, thereby ensuring that the decisions or suggestions made have sufficient basis and authority.

[0070] The specific method of using a large number of specific question-answer data pairs to construct a preset knowledge base includes: first, extracting the specific question parts from all the collected specific question-answer data pairs, and then performing text preprocessing on all the extracted specific questions, including removing punctuation marks, stop words, numbers and other irrelevant information, and performing word segmentation, word form restoration and other processing to obtain text data suitable for word embedding model processing; then, using the BGE model or other language model as a text vector model, the preprocessed text data is input into the BGE model to obtain the vector representation of each word or phrase in the preprocessed specific question in the high-dimensional space. These vectors can capture the semantic relationship between words or phrases, and combine the vectors of all words or phrases in some form (such as averaging, summing, weighted summing, etc.) to obtain the text vector of the entire specific question, that is, to obtain the first text vector used to characterize the semantic features of the specific question.

[0071] Then, after obtaining all the first text vectors, all the first text vectors are stored in the FAISS vector library. Each first text vector corresponds to a related document and a corresponding answer. In the FAISS vector library, the association between the first text vector and the related documents and answers usually exists in the form of meta information, such as a document ID or an answer ID, which is used to find the actual document and answer content in the preset knowledge base. The preset knowledge base includes the FAISS vector library and other forms of data storage and retrieval mechanisms. For example, the preset knowledge base stores the first text vector and meta information, as well as the documents, answers, laws and regulations corresponding to the first text vector. The content stored in the preset knowledge base is usually stored in a database, file system or other form for easy retrieval and display.

[0072] In this embodiment, by converting a specific question into a first text vector, and storing each first text vector, a document related to the first text vector, and an answer corresponding to the first text vector in a preset knowledge base, the speed and accuracy of vector operations can be used to quickly locate a specific question-answer data pair similar to a new specific question encountered in a specific job in the specific knowledge base, so as to improve retrieval efficiency and reduce the waiting time of a specific person when querying information. That is, when a new specific question generated in a specific job is queried in the preset knowledge base, the specific question related to the new specific question that already exists in the preset knowledge base can be found by calculating the similarity between the text vector corresponding to the new specific question and the first text vector stored in the preset knowledge base. Once the specific question related to the new specific question that already exists in the preset knowledge base is found, the document associated with the new specific question and the corresponding answer can be directly extracted from the preset knowledge base to make a decision on the new specific question.

[0073] In some embodiments, Figure 2FIG. 2 shows a flow chart of another question-answering model training method provided by an embodiment of the present application. Figure 2 As shown, before step S104, the question-answering model training method also includes steps S101 to S103.

[0074] Step S101: for an input specific question, searching a preset knowledge base for at least one document related to the input specific question.

[0075] Specifically, when an input specific question is obtained, the input specific question may be a new specific question, that is, it may not have a corresponding relevant document and a corresponding answer in the preset knowledge base. Due to the diversity of natural language texts and the richness of semantics, the new specific question may have a semantic association or similarity with certain specific questions in the preset knowledge base. Therefore, the text vectorization and similarity search technology can be used to search for at least one specific question related to the input specific question from the preset knowledge base according to the input specific question, and the relevant documents and corresponding answers corresponding to at least one specific question related to the input specific question, so as to provide relevant context information and reference answers for answering the input specific question.

[0076] Specifically, searching for at least one document related to the input specific question from the preset knowledge base according to the input specific question includes: first, performing text preprocessing on the input specific question, including removing irrelevant information such as punctuation marks, stop words, numbers, and performing word segmentation, word form restoration, etc., so as to convert the input specific question into a text form that is more conducive to the understanding of the preset language model; then, using the BGE model to vectorize the preprocessed input specific question. The BGE model converts the input specific question into a high-dimensional vector, namely the second text vector, which can characterize the semantic features of the input specific question; after obtaining the second text vector, respectively calculate the similarity between the second text vector and all the first text vectors in the preset knowledge base, and obtain the similarity score corresponding to the second text vector and each first text vector, so as to find the specific question related to the input specific question from the preset knowledge base.

[0077] Finally, the specific question related to the input specific question, the documents related to the specific question related to the input specific question, and the corresponding answers, that is, the specific question-answer data pairs related to the input specific question, are determined based on the similarity scores. Specifically, once the similarity scores corresponding to the second text vector and each of the first text vectors are calculated, the specific question-answer data pairs related to the input specific question can be determined based on these similarity scores. First, the calculated similarity scores are sorted from high to low, so that the specific questions most relevant to the input specific question will be ranked at the front. Afterwards, the top k specific questions, related documents, and corresponding answers ranked high in similarity scores are filtered out from the sorting results according to the actual situation, and the filtered results are used as the specific question-answer data pairs related to the input specific question.

[0078] Step S102: Input the input specific question, the searched documents related to the input specific question and the preset guidance text into the preset language model to obtain an answer with referenced documents corresponding to the input specific question. The guidance text is used to guide the result output method of the preset language model.

[0079] Specifically, after obtaining the specific question-answer data pairs related to the input specific question, the relevant documents and preset guidance texts (i.e., the corresponding answers) corresponding to each specific question-answer data pair related to the input specific question are extracted. When there are k specific question-answer data pairs related to the input specific question, there are also k corresponding relevant documents and preset guidance texts. The value of k can be determined according to actual conditions. For example, k is 8. Afterwards, the input specific question, the first k documents related to the input specific question, and the preset guidance texts contained in each relevant document are input into the preset language model. The preset language model understands and analyzes the input data, and summarizes and concludes based on the results of the understanding and analysis, so as to generate answer information with referenced documents corresponding to the input specific question.

[0080] Specifically, the preset language model can be a ChatGPT model or other language model. The input specific question, the top k documents related to the input specific question, and the preset guidance text contained in each related document are input into the ChatGPT model or other language model. The ChatGPT model or other language model uses its powerful natural language processing capabilities and deep learning algorithms to conduct in-depth analysis and understanding of the input specific question, the top k related documents, and the corresponding preset guidance text. In the process of generating answers, ChatGPT will pay special attention to the use of relevant documents. That is, key information, legal terms or actual cases are extracted from relevant documents, and the key information, legal terms or actual cases extracted from relevant documents are directly integrated into the answer to generate a detailed, authoritative and grammatically compliant answer information. The answer information not only contains the direct answer to the input specific question, but also contains the background, evidence documents and contextual reference documents related to the input specific question, making the answer more convincing and credible.

[0081] Step S103: Generate a specific enhanced sample based on the input specific question and the corresponding answer with the reference document.

[0082] Specifically, each input specific question and the answer with reference document generated by the preset language model constitute a specific enhanced sample. When a new specific problem is encountered in a specific work, a specific enhanced sample including the new specific question and the corresponding answer with reference document can be obtained according to the above steps S101 to S103.

[0083] In a specific example, Figure 3 FIG. 1 is a flow chart showing a specific example of generating a specific enhanced sample provided by an embodiment of the present application. Figure 3 As shown in the figure, the specific question (i.e., the query question) entered is "A production and operation unit has more than 100 employees, but the unit does not have a production safety management organization or full-time production safety management personnel, and does not conduct production safety education and training. In this case, is it a violation of the Production Safety Law?"

[0084] According to the input specific question, the first 8 specific questions (not shown) related to the input specific question are searched from the specific question-answer data pairs stored in the preset knowledge base, the corresponding related documents and the preset guidance text in each related document. The corresponding related documents are document [1], document [2], document [3], document [4], document [5], document [6], document [7] and document [8].

[0085] The default guidance text corresponding to document [1] (i.e. the answer to the specific question in document [1]) is: Article 24 of the Work Safety Law (2021) Mines, metal smelting, construction, transportation units and production, operation, storage and loading and unloading units of dangerous goods shall set up work safety management organizations or assign full-time work safety management personnel. Other production and operation units other than those specified in the preceding paragraph shall set up work safety management organizations or assign full-time work safety management personnel if they have more than 100 employees; if they have less than 100 employees, they shall assign full-time or part-time work safety management personnel.

[0086] The default guidance text corresponding to document [2] is: Article 107 of the Work Safety Law (2021): If employees of a production and operation unit fail to fulfill their job safety responsibilities, disobey management, or violate work safety rules and regulations or operating procedures, the production and operation unit shall criticize and educate them and impose penalties in accordance with relevant rules and regulations; if a crime is constituted, criminal liability shall be pursued in accordance with relevant provisions of the Criminal Law.

[0087] The default guidance text corresponding to document [3] is: Article 25 of the Work Safety Law (2021) stipulates that the work safety management organization and work safety management personnel of a production and operation unit shall perform the following duties:

[0088] (1) Organizing or participating in the formulation of the unit's production safety rules and regulations, operating procedures and emergency rescue plans for production safety accidents;

[0089] (2) Organizing or participating in the unit's production safety education and training, and truthfully recording the production safety education and training;

[0090] (3) Organize the identification and assessment of hazardous sources, and supervise the implementation of safety management measures for major hazardous sources in the unit;

[0091] (4) Organizing or participating in emergency rescue drills in the unit;

[0092] (5) Inspect the production safety status of the unit, promptly identify potential production safety accident hazards, and make suggestions for improving production safety management;

[0093] (6) To stop and correct illegal commands, orders to conduct risky operations, and violations of operating procedures;

[0094] (VII) Supervise the implementation of the unit’s production safety rectification measures.

[0095] Production and operation units may set up full-time persons in charge of production safety to assist the unit’s main persons in charge in performing their production safety management duties.

[0096] The default guidance text corresponding to document [4] is: Article 20 of the Occupational Safety and Health Law (2021) Production and operation units shall meet the safety production conditions stipulated in this Law and relevant laws, administrative regulations or industry standards; if they do not meet the safety production conditions, they shall not engage in production and operation activities.

[0097] The default guidance text corresponding to document [5] is: According to Article 83 of my country's "Work Safety Law", if a production and operation unit commits any of the following acts and causes serious consequences and constitutes a crime, it shall be held criminally liable in accordance with the relevant provisions of the Criminal Law: (1) A mining construction project or a construction project for the production or storage of dangerous goods does not have a safety facility design or the safety facility design is not submitted to the relevant departments for review and approval in accordance with the regulations; (2) The construction unit of a mining construction project or a construction project for the production or storage of dangerous goods fails to construct in accordance with the approved safety facility design; (3) Before a mining construction project or a construction project for the production or storage of dangerous goods is completed and put into production or use, Safety facilities have not been inspected and accepted; (4) Obvious safety warning signs have not been set up in production and operation sites and related facilities and equipment with major risk factors; (5) The installation, use, inspection, modification and scrapping of safety equipment do not meet industry standards; (6) Safety equipment has not been regularly maintained, serviced and regularly inspected; (7) Labor protection equipment that meets industry standards has not been provided to employees; (8) Special equipment and containers and transportation vehicles for dangerous goods have not been inspected and tested by professionally qualified institutions and have not obtained safety use certificates or safety signs before being put into use; (9) Use of processes and equipment that endanger production safety and are expressly eliminated or prohibited from use. Of course, if the above-mentioned acts occur but do not cause serious consequences, they do not constitute a crime. The corresponding punishment is to order rectification within a time limit. If rectification is not made within the time limit, the construction shall be stopped or production and business shall be suspended for rectification, and a fine of up to RMB 50,000 may also be imposed.

[0098] The default guidance text corresponding to document [6] is: Article 22 of the Occupational Safety and Health Law (2021) stipulates that the all-staff safety responsibility system of production and operation units shall clearly define the responsible persons, scope of responsibility and assessment standards for each position.

[0099] Production and operation units should establish corresponding mechanisms, strengthen supervision and assessment of the implementation of the safety production responsibility system for all employees, and ensure the implementation of the safety production responsibility system for all employees.

[0100] The default guidance text corresponding to document [7] is: Article 81 of the Production Safety Law (2021) Production and operation units shall formulate emergency rescue plans for production safety accidents in their units, connect them with the emergency rescue plans for production safety accidents formulated by the organizers, and organize drills regularly.

[0101] The default guidance text corresponding to document [8] is: Article 96 of the Work Safety Law (2021) If other persons in charge and work safety management personnel of a production and operation unit fail to perform their work safety management duties as prescribed by this Law, they shall be ordered to correct the situation within a time limit and be fined not less than RMB 10,000 but not more than RMB 30,000; if it causes a production safety accident, their qualifications related to work safety shall be suspended or revoked, and they shall be fined not less than 20% but not more than 50% of their annual income in the previous year; if it constitutes a crime, they shall be held criminally liable in accordance with the relevant provisions of the Criminal Law.

[0102] After obtaining 8 relevant documents corresponding to the specific question related to the input specific question and the preset guidance text in each relevant document, the input specific question, the 8 relevant documents and the preset guidance text in each relevant document are input into the ChatGPT model. The ChatGPT model understands and analyzes the input specific question, the 8 relevant documents and the preset guidance text in each relevant document, and summarizes and summarizes the results of the understanding and analysis to generate an answer containing the referenced document: "Yes, the production and operation unit has violated the Occupational Safety and Health Law. According to Article 24 of the Occupational Safety and Health Law, a production and operation unit with more than 100 employees shall set up a production safety management organization or equip full-time safety and health personnel. production management personnel[1]. Therefore, the unit has violated the law by not setting up a production safety management organization or equipping full-time production safety management personnel. \n\nIn addition, according to Article 25 of the "Work Safety Law", the production safety management organization or production safety management personnel of a production and operation unit shall perform the duties of organizing or participating in the production safety education and training of the unit[3]. The unit's failure to conduct production safety education and training also violated this legal provision. If the unit causes a production safety accident due to violation of the Production Safety Law, other responsible persons and production safety management personnel may be punished according to Article 96 of the "Work Safety Law"[8]. In some serious cases, it may also constitute a crime and be held criminally liable according to the Criminal Law[5]. "

[0103] It can be seen that the ChatGPT model obtains answer information containing reference documents by understanding and analyzing the input specific question and the corresponding preset guidance text in documents [1] to [8]. The answer information containing reference documents not only gives a correct answer to the question of whether the input specific question is illegal, but also quotes the contents of documents [1], [3], [5] and [8] as the judgment basis and evidence documents for the decision, thereby increasing the credibility and authority of the answer to the decision.

[0104] Specifically, when each input specific question and its corresponding answer information containing reference documents are generated, they are regarded as specific enhancement samples. As these specific enhancement samples continue to accumulate, when a certain number of specific enhancement samples are accumulated and a sufficient number of specific enhancement samples are obtained, the sufficient number of specific enhancement samples can be used as the first question-answering dataset.

[0105] Step S105: Use the specific questions in the specific enhanced samples and the corresponding answers with referenced documents as model input samples and model output samples respectively, perform enhanced training on the basic question-answering model, and obtain a target question-answering model; the basic question-answering model is trained based on the second question-answering data set, and the second question-answering data set includes multiple questions and answers corresponding to each question.

[0106] Specifically, the basic question-and-answer model is universal and is trained by a second question-and-answer data set consisting of questions and corresponding answers collected from multiple sources, wherein the questions in the second question-and-answer data set include but are not limited to various fields such as daily life, science and technology, culture, history, and specific fields. In a specific example, questions and corresponding answers in non-specific fields such as daily life, science and technology, culture, and history are used as universal answer data pairs, and questions and corresponding answers in specific fields are used as specific question-and-answer data pairs. When the second question-and-answer data set is used to train the basic question-and-answer model, the training data contained in the second question-and-answer data set includes universal answer data pairs and specific question-and-answer data pairs. Therefore, the basic question-and-answer model has universal question-and-answer capabilities and specific question-and-answer capabilities.

[0107] After the basic question-answering model is generated based on questions in various fields and the corresponding answers, since the basic question-answering model only has general question-answering capabilities and specific question-answering capabilities, and does not have the ability to generate answers with reference documents corresponding to questions in specific fields, after inputting specific questions into the basic question-answering model, only the answers corresponding to the specific questions can be obtained, and it is impossible to generate answer information with reference documents for the specific questions that meets specific requirements and has authority. Therefore, after obtaining the first question-answering dataset constructed by specific enhanced samples, the specific enhanced samples in the first question-answering dataset can be used to further perform enhanced training on the basic question-answering model, that is, the specific question model in the specific enhanced samples is input as the sample, and the answer with reference documents corresponding to the specific question in the specific enhanced samples is used as the model output sample, and the basic question-answering model is further enhanced and trained, so that the basic question-answering model after enhanced training (i.e., the target question-answering model) has the ability to generate answer information with reference documents.

[0108] In some embodiments, obtaining the first question-answering dataset includes:

[0109] Step S1041: Based on the number of questions included in the second question and answer data set, a corresponding number of specific enhanced samples are obtained in a preset ratio to obtain a first question and answer data set, wherein the number of specific enhanced samples included in the first question and answer data set is less than the number of questions included in the second question and answer data set.

[0110] In a specific example, questions in a specific field and the corresponding answers with reference documents are used as specific enhanced question-and-answer data pairs, the above-mentioned general answer data pairs, specific question-and-answer data pairs and specific enhanced question-and-answer data pairs are used as three different types of sample data, and the sample data containing a corresponding number of general answer data pairs and specific question-and-answer data pairs is used as the second question-and-answer data set. The second question-and-answer data set is used to train the basic question-and-answer model, so that the basic question-and-answer model has general question-and-answer and specific question-and-answer capabilities. Then, a corresponding number of specific enhanced question-and-answer data pairs are obtained at a preset ratio, and the specific enhanced samples containing a corresponding number of specific enhanced question-and-answer data pairs are used as the first question-and-answer data set. The first question-and-answer data set is used to further enhance the training of the basic question-and-answer model, so that the enhanced trained basic question-and-answer model has the ability to generate more authoritative answer information with reference documents for specific questions.

[0111] Exemplarily, the preset ratio of the number of general answer data pairs, the number of specific question and answer data pairs, and the number of specific enhanced question and answer data pairs is set to 2:1:1, that is, the preset ratio of the number of sample data in the second question and answer data set to the number of sample data in the first question and answer data set is set to 3:1. By obtaining a corresponding number of specific enhanced samples at the preset ratio of 3:1, it is ensured that while the model is exposed to a sufficient number of specific enhanced samples so that the model has the ability to generate more authoritative answer information with reference documents for specific questions, it does not focus too much on the characteristics of the specific enhanced samples, so as to maintain the model's original general question and answer capabilities and specific question and answer capabilities.

[0112] In some embodiments, Figure 4 FIG. 1 is a flow chart showing the sub-steps of step S105 provided in an embodiment of the present application. Figure 4 As shown, the specific question in the specific enhanced sample and the corresponding answer with the reference document are used as the model input sample and the model output sample respectively, and the basic question-answering model is enhanced trained, including steps S1051 to S1053.

[0113] Step S1051: Input the specific question in the specific enhanced sample into the basic question-answering model to obtain the initial predicted answer output by the basic question-answering model.

[0114] Specifically, after obtaining a corresponding number of specific enhanced samples, the specific questions in each specific enhanced sample are input one by one into the basic question and answer model trained using the second question and answer data set. The basic question and answer model uses the model parameters obtained by the current training to encode the input specific questions and convert them into a numerical form that the model can understand. After that, the model decodes or infers the encoded results to generate an initial predicted answer for the input specific question. At this time, since the basic question and answer model may not be sufficiently trained in a specific field, this initial predicted answer may not be completely accurate or comprehensive.

[0115] Step S1052: Calculate the loss between the initial prediction answer and the corresponding model output sample.

[0116] Specifically, in order to evaluate the gap between the initial predicted answer generated by the model and the answer with the reference document provided in the specific enhanced sample (i.e., the true answer), the initial predicted answer is compared with the answer with the reference document provided in the specific enhanced sample, and the loss between the initial predicted answer and the answer with the reference document provided in the specific enhanced sample is calculated, for example, the cross entropy loss between the initial predicted answer and the answer with the reference document provided in the specific enhanced sample is calculated to obtain the difference between the prediction result of the model and the true answer.

[0117] Step S1053: Iteratively update the model parameters of the basic question-answering model according to the loss between the initial predicted answer and the corresponding model output sample until the basic question-answering model meets the preset second training stop condition.

[0118] Specifically, according to the calculated loss value between the initial predicted answer and the corresponding model output sample, the loss value is back-propagated to each layer of the basic question-answering model through the back-propagation algorithm, so as to calculate the gradient of the parameters of each layer. Then, the parameters of the basic question-answering model are updated according to these gradients using an optimization algorithm (such as Adam, SGD, etc.). The optimization algorithm adjusts the value of the parameter to minimize the loss between the initial predicted answer and the corresponding model output sample, and the process of step S1051 to step S1053 is repeated multiple times until the basic question-answering model meets the preset second training stop condition. Among them, the second training stop condition includes: when the change in the loss between the initial predicted answer and the corresponding model output sample is less than a certain threshold, that is, the basic question-answering model has converged and the training stops; or, when the basic question-answering model is undergoing enhanced training, if the training round reaches the preset maximum training round, the training stops.

[0119] In some embodiments, Figure 5 FIG. 2 shows a flow chart of another question-answering model training method provided by an embodiment of the present application. Figure 5As shown, after step S105, the question-answering model training method also includes step S106 and step S107.

[0120] Step S106: Obtain a preference data set; the preference data set includes positive sample data and negative sample data; the positive sample data includes specific question samples and corresponding reference answers, and the reference answers come with reference documents; the negative sample data includes specific question samples and answers output by the target question-answering model based on the specific question samples.

[0121] Specifically, after using the first question-answering dataset to further enhance the training of the basic question-answering model so that the target question-answering model has the ability to generate more authoritative answer information with reference documents for specific questions, the target question-answering model still has a high probability of generating incorrect answers. Therefore, after obtaining the target question-answering model, it is also necessary to obtain the preference dataset in order to use the preference dataset to optimize the target question-answering model so that the optimized target question-answering model can generate answer information with reference documents with higher accuracy.

[0122] The preference data set includes positive sample data and negative sample data. The positive sample data contains specific question samples and corresponding reference answers, and the reference answers come with reference documents; the negative sample data contains specific question samples and the answers output by the target question-answering model based on these question samples, which are likely to be wrong answers.

[0123] Step S107: Based on the preference data set, preference alignment optimization training is performed on the target question answering model to obtain an optimized target question answering model.

[0124] Among them, preference alignment optimization is a method widely used in large-scale language models, which aims to make the output of the model more in line with human preferences. Specifically, preference alignment optimization adjusts and optimizes the parameters and behaviors of the model through a series of technical means and strategies to generate more accurate, reasonable, useful and human-expected answers. Preference alignment optimization methods include human preference optimization algorithm (Reinforcement Learning from Human Feedback, RLHF), direct preference optimization (Direct Preference Optimization, DPO), identity preference optimization (Identity Preference Optimisation, IPO) and odds ratio optimization (Odds Ratio Preference Optimization, ORPO) and other methods. The embodiment of the present application can select a suitable preference alignment optimization method according to actual conditions.

[0125] In a specific example, since the ORPO optimization method has the characteristics of lower cost than the RLHF optimization method, and is simpler to operate than the DPO optimization method and the IPO optimization method, and does not require the training of additional preference models, the embodiment of the present application can select the ORPO optimization method to perform preference alignment optimization on the target question-answering model. Specifically, the ORPO optimization method achieves preference alignment by combining instruction fine-tuning and preference alignment into a single process and introducing OR loss in the loss function, thereby avoiding the training of additional preference models and simplifying the operation process.

[0126] In some embodiments, Figure 6 FIG. 1 is a flow chart showing the sub-steps of step S107 provided in an embodiment of the present application. Figure 6 As shown, step S107 further includes steps S1071 to S1074.

[0127] Step S1071: Input the specific question sample in the positive sample data into the current target question-answering model, obtain the prediction result corresponding to the positive sample data, and calculate the first loss between the prediction result corresponding to the positive sample data and the reference answer.

[0128] In a specific example, when the ORPO optimization method is selected to perform preference alignment optimization on the target question-answering model, the input positive sample data (including specific question samples and their corresponding reference answers) is used to calculate the difference between the model prediction results and the reference answers.

[0129] Specifically, the specific question samples in each positive sample data are input one by one into the current target question and answer model that has been obtained through enhanced training of the first question and answer data set. The current target question and answer model uses the model parameters obtained by the current training to encode the specific question samples in the input positive sample data, and converts them into a numerical form that the current target question and answer model can understand. Thereafter, the current target question and answer model decodes or infers the encoded results to generate corresponding prediction results for the specific question samples in the input positive sample data. At this time, since the current target question and answer model may not fully cover all situations during the training process, the prediction results generated for the specific question samples in the input positive sample data may not be completely accurate or comprehensive. In order to evaluate the gap between the prediction result generated by the current target question answering model for a specific question sample in the input positive sample data and the corresponding reference answer, the prediction result generated for the specific question sample in the input positive sample data is compared with the corresponding reference answer, and the loss between the prediction result generated for the specific question sample in the input positive sample data and the corresponding reference answer is calculated, for example, the cross entropy loss between the prediction result generated for the specific question sample in the input positive sample data and the corresponding reference answer is calculated to obtain the difference between the prediction result generated by the current target question answering model for the specific question sample in the input positive sample data and the corresponding reference answer, that is, the first loss

[0130] Step S1072: Input the specific question sample in the negative sample data into the current target question-answering model, obtain the prediction result corresponding to the negative sample data, and calculate the second loss between the prediction result corresponding to the negative sample data and the corresponding answer in the negative sample data.

[0131] In a specific example, when the ORPO optimization method is selected to perform preference alignment optimization on the target question-answering model, the input negative sample data (including specific question samples and the answers output by the target question-answering model based on the specific question samples) is also used to calculate the OR loss (Odds Ratio Loss, OR). The OR loss is used to weakly punish unpopular generated content and strongly reward popular generated content.

[0132] Specifically, the specific question samples in each negative sample data are input one by one into the current target question answering model that has been trained through the first question answering data set enhancement. The current target question answering model uses the model parameters obtained from the current training to encode the specific question samples in the input negative sample data and convert them into a numerical form that the current target question answering model can understand. After that, the current target question answering model decodes or infers the encoded results to generate corresponding prediction results for the specific question samples in the input negative sample data. Next, the OR loss between the corresponding prediction results generated for the specific question samples in the input negative sample data and the answers provided in the negative sample data is calculated, that is, the second loss

[0133] In a specific example, calculating the second loss between the prediction result corresponding to the negative sample data and the corresponding answer in the negative sample data includes steps S10721 to S10724.

[0134] Step S10721: For a specific question sequence x in the input negative sample data, calculate the average logarithmic probability logP of the output sequence y (containing m tokens) corresponding to the sequence x generated by the current target question-answering model θ (y|x) is:

[0135]

[0136] Among them, θ represents the parameters of the current target question answering model, y t Represents the tth token in the output sequence y, y ＜t Indicates that the t All previous tokens; logP θ (y|x) is used to measure the probability of the current target question-answering model generating a given sequence y.

[0137] Step S10722: Calculate the ratio of the conditional probability of the current target question-answering model generating a given sequence y to the conditional probability of not generating the given sequence y (i.e. generating any other sequence) oddS θ (y|x) is:

[0138]

[0139] In this embodiment, oddS θ (y|x) is the ratio between the probability of the current target question-answering model generating the target sequence y and the probability of generating all other sequences; oddS θ (y|x) reflects the preference of the current target question answering model for generating the target sequence y.

[0140] Step S10723: Calculate the positive sample sequence y generated by the current target question-answering modelw And generate negative sample sequence y l The odds ratio (OR) θ (y w ,y l )for:

[0141]

[0142] Among them, oddS θ (y w |x) means that under the condition of given model parameters θ and input sequence x, the current target question answering model generates a positive sample sequence y w The ratio of oddS θ (y l |x) means that under the condition of given model parameters θ and input sequence x, the current target question answering model generates a negative sample sequence y l Ratio of θ (y w ,y l ) measures the preference difference between positive and negative samples of the current target question answering model.

[0143] Step S10724: Define the second loss for:

[0144]

[0145] Where σ represents the sigmoid function.

[0146] Step S1073: Perform a weighted summation on the first loss and the second loss to obtain a target loss, wherein the weight of the second loss is less than the weight of the first loss.

[0147] In a specific example, define the target loss L ORPO for:

[0148]

[0149] in, For the first loss, is the second loss, and λ is a hyperparameter used to balance the first loss and the second loss. For (x,y w ,y l ) to calculate L for all possible data points on ORPO The average or expected value.

[0150] In this embodiment, since the target loss of the ORPO optimization method includes both the first loss and the second loss, after obtaining the first loss And the second loss After that, in order to balance the first loss and the second loss By taking the weighted sum of the first loss and the second loss, we can get the target loss L oRPO In this embodiment, the weight of the second loss is set to be smaller than the weight of the first loss. Exemplarily, the weight of the first loss is set to 1, and the weight of the second loss is set to 0.1.

[0151] It should be noted that setting the weight of the first loss to 1 and setting the weight of the second loss to 0.1 is only a specific example. In this embodiment, the weights of the first loss and the second loss can be adjusted according to actual conditions.

[0152] Step S1074: Iteratively update the model parameters of the current target question-answering model according to the target loss until the target question-answering model meets the preset first training stop condition.

[0153] Specifically, according to the calculated target loss, the target loss is back-propagated to each layer of the current target question-answering model through the back-propagation algorithm, so as to calculate the gradient of the parameters of each layer. Then, the optimization algorithm (such as Adam, SGD, etc.) is used to update the parameters of the current target question-answering model according to these gradients. The optimization algorithm adjusts the value of the parameter to minimize the target loss, and the process of step S1071 to step S1074 will be repeated many times until the current target question-answering model meets the preset first training stop condition. Among them, the first training stop condition includes: when the change in the target loss is less than a certain threshold, that is, the current target question-answering model has converged, and the training stops; or, when the target question-answering model is performing preference alignment optimization training, if the training round reaches the preset maximum training round, the training stops.

[0154] In a specific example, if Figure 7 As shown, in the ORPO optimization method, two samples are used as input data, which are recorded as positive sample data and negative sample data respectively, wherein the two sample data correspond to the same specific question, but the answers to the specific question are different, the answer to the positive sample data corresponds to the expected answer, and the answer to the negative sample data corresponds to the unexpected answer. In the ORPO optimization process, the goal is to adjust the target loss of the target question-answering model so that the target question-answering model is more inclined to the answer corresponding to the positive sample data when outputting the predicted answer. Among them, the answer corresponding to the positive sample data can come from the reference answer that is correctly labeled for the specific question; or, it can come from the answer generated by the target question-answering model using a language model such as the ChatGPT model, and the corrected answer is used as the answer corresponding to the positive sample data. The answer corresponding to the negative sample data can come from the answer generated by the target question-answering model.

[0155] Figure 8FIG. 1 is a flow chart of a question-answering method provided in an embodiment of the present application. Figure 8 As shown, the question-answering method includes steps S201 to S202.

[0156] Step S201: Obtain the target question.

[0157] When new specific problems arise in specific work, they will be used as target specific problems. Specifically, new specific problems can be obtained from news media, social media and other channels in a timely manner; or new specific problems can be obtained in a timely manner through regular social surveys, public opinion surveys, public security situation assessments, etc.

[0158] Step S202: Input the target question into the target question-answering model to obtain an answer with referenced documents corresponding to the target question. The target question-answering model is trained using the above-mentioned question-answering model training method.

[0159] The question-and-answer method provided in the embodiment of the present application can utilize the target question-and-answer model to generate an answer with referenced documents corresponding to the target question when a specific person is dealing with a target question, and can make more intelligent and accurate decisions based on the answer with referenced documents generated by the target question-and-answer model, so as to improve the credibility and persuasiveness of the decisions made by the specific person when dealing with the target question.

[0160] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0161] Corresponding to the specific question-answer model creation method described in the above embodiment, Fig. 9 This is a structural block diagram of the question-answering model training device provided in an embodiment of the present application. For the sake of ease of explanation, only the parts related to the embodiment of the present application are shown.

[0162] Reference Fig. 9 , the question answering model training device includes:

[0163] The first acquisition module 91 is used to acquire a first question-answering dataset; the first question-answering dataset includes more than two specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question.

[0164] The enhanced training module 92 is used to use the specific question in the specific enhanced sample and the corresponding answer with the reference document as the model input sample and the model output sample respectively, to perform enhanced training on the basic question and answer model to obtain a target question and answer model; the basic question and answer model is trained based on the second question and answer data set, and the second question and answer data set includes multiple questions and answers corresponding to each of the questions.

[0165] In some embodiments, the question-answering model training apparatus further comprises:

[0166] The search module 93 is used to search for at least one document related to the input specific question from a preset knowledge base.

[0167] The input-output module 94 is used to input the specific question, the searched documents related to the specific question and the preset guidance text into the preset language model to obtain the answer with the referenced document corresponding to the specific question, and the guidance text is used to guide the result output method of the preset language model.

[0168] The specific enhanced sample generating module 95 is used to generate the specific enhanced sample based on the specific question and the corresponding answer with the reference document.

[0169] In some embodiments, the question-answering model training apparatus further comprises:

[0170] The preference data set acquisition module 96 is used to acquire a preference data set; the preference data set includes positive sample data and negative sample data; the positive sample data includes a specific question sample and a corresponding reference answer, and the reference answer comes with a reference document; the negative sample data includes the specific question sample and the answer output by the target question-answering model based on the specific question sample.

[0171] The preference alignment optimization training module 97 is used to perform preference alignment optimization training on the target question-answering model based on the preference data set to obtain an optimized target question-answering model.

[0172] In some embodiments, the preference alignment optimization training module 97 is further configured to:

[0173] Input a specific question sample in the positive sample data into the current target question-answering model to obtain a prediction result corresponding to the positive sample data, and calculate a first loss between the prediction result corresponding to the positive sample data and the reference answer.

[0174] The specific question sample in the negative sample data is input into the current target question-answering model to obtain the prediction result corresponding to the negative sample data, and the second loss between the prediction result corresponding to the negative sample data and the corresponding answer in the negative sample data is calculated.

[0175] A weighted sum is performed on the first loss and the second loss to obtain a target loss, wherein the weight of the second loss is less than the weight of the first loss.

[0176] Iteratively update the model parameters of the current target question-answering model according to the target loss until the target question-answering model meets a preset first training stop condition.

[0177] In some embodiments, the first acquisition module 91 is further used for:

[0178] Based on the number of questions included in the second question and answer dataset, a corresponding number of specific enhanced samples are obtained in a preset ratio to obtain the first question and answer dataset, wherein the number of specific enhanced samples included in the first question and answer dataset is less than the number of questions included in the second question and answer dataset.

[0179] In some embodiments, the enhanced training module 92 is further configured to:

[0180] The specific question in the specific enhanced sample is input into the basic question-answering model to obtain an initial predicted answer output by the basic question-answering model.

[0181] Calculate the loss between the initial prediction answer and the corresponding output sample of the model.

[0182] The model parameters of the basic question-answering model are iteratively updated according to the loss between the initial predicted answer and the corresponding model output sample until the basic question-answering model meets a preset second training stop condition.

[0183] Corresponding to the question-answering method described in the above embodiment, Fig.10 This is a structural block diagram of the question-and-answer device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0184] Reference Fig.10 , the question-answering device comprises:

[0185] The second acquisition module 101 is used to acquire a target question.

[0186] The generation module 102 is used to input the target question into the target question-answering model to obtain an answer with a reference document corresponding to the target question, wherein the target question-answering model is trained using the above-mentioned question-answering model training method.

[0187] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0188] in addition, Fig. 9 The question-answering model training device shown in Fig.10 The question-answering device shown may be a software unit, a hardware unit, or a combination of software and hardware, which is built into an existing terminal device, or may be integrated into the terminal device as an independent hanger, or may exist as an independent terminal device.

[0189] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0190] Fig.11 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig.11 As shown, the terminal device 11 of this embodiment includes: at least one processor 110 ( Fig.11 Only one processor is shown in the figure), a memory 111, and a computer program 112 stored in the memory 111 and executable on the at least one processor 110, wherein the processor 110 implements the steps of any of the above-mentioned question-answering model training method / question-answering method embodiments when executing the computer program 112.

[0191] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Fig.11 It is only an example of the terminal device 11 and does not constitute a limitation on the terminal device 11. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0192] The processor 110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0193] In some embodiments, the memory 111 may be an internal storage unit of the terminal device 11, such as a hard disk or memory of the terminal device 11. In other embodiments, the memory 111 may also be an external storage device of the terminal device 11, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 11. Further, the memory 111 may also include both an internal storage unit of the terminal device 11 and an external storage device. The memory 111 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 111 may also be used to temporarily store data that has been output or is to be output.

[0194] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0195] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0196] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0197] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0198] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0199] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A question-answering model training method, characterized in that: include: Get the first question-answering dataset; The first question-answering dataset includes more than two specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question; The specific question in the specific enhanced sample and the corresponding answer with the reference document are used as the model input sample and the model output sample respectively, and the basic question and answer model is enhanced trained to obtain the target question and answer model; the basic question and answer model is trained based on the second question and answer data set, and the second question and answer data set includes multiple questions and answers corresponding to each of the questions.

2. The question-answering model training method according to claim 1, characterized in that: Before obtaining the first question-answering dataset, it also includes: For the input specific question, searching a preset knowledge base for at least one document related to the specific question; Inputting the specific question, the searched documents related to the specific question and the preset guidance text into a preset language model to obtain an answer with referenced documents corresponding to the specific question, wherein the guidance text is used to guide the result output mode of the preset language model; The specific enhanced sample is generated based on the specific question and the corresponding answer with the reference document.

3. The question-answering model training method according to claim 1 or 2, characterized in that: After obtaining the target question answering model, it also includes: Obtain a preference data set; the preference data set includes positive sample data and negative sample data; the positive sample data includes a specific question sample and a corresponding reference answer, and the reference answer has a reference document; the negative sample data includes the specific question sample and an answer output by the target question-answering model based on the specific question sample; Based on the preference data set, preference alignment optimization training is performed on the target question answering model to obtain an optimized target question answering model.

4. The question-answering model training method according to claim 3, characterized in that: The performing preference alignment optimization training on the target question answering model based on the preference data set includes: Inputting a specific question sample in the positive sample data into the current target question-answering model, obtaining a prediction result corresponding to the positive sample data, and calculating a first loss between the prediction result corresponding to the positive sample data and the reference answer; Inputting a specific question sample in the negative sample data into the current target question-answering model, obtaining a prediction result corresponding to the negative sample data, and calculating a second loss between the prediction result corresponding to the negative sample data and a corresponding answer in the negative sample data; Performing a weighted summation on the first loss and the second loss to obtain a target loss, wherein the weight of the second loss is less than the weight of the first loss; Iteratively update the model parameters of the current target question-answering model according to the target loss until the target question-answering model meets a preset first training stop condition.

5. The question-answering model training method according to claim 1 or 2, characterized in that: The obtaining of the first question-answering data set comprises: Based on the number of questions included in the second question and answer dataset, a corresponding number of specific enhanced samples are obtained in a preset ratio to obtain the first question and answer dataset, wherein the number of specific enhanced samples included in the first question and answer dataset is less than the number of questions included in the second question and answer dataset.

6. The question-answering model training method according to claim 5, characterized in that: The specific question in the specific enhanced sample and the corresponding answer with the reference document are used as a model input sample and a model output sample respectively to perform enhanced training on the basic question-answering model, including: Inputting the specific question in the specific enhanced sample into the basic question-answering model to obtain an initial predicted answer output by the basic question-answering model; Calculating the loss between the initial prediction answer and the corresponding model output sample; The model parameters of the basic question-answering model are iteratively updated according to the loss between the initial predicted answer and the corresponding model output sample until the basic question-answering model meets a preset second training stop condition.

7. A question-answering method, characterized in that: include: Get the target question; The target question is input into a target question-answering model to obtain an answer with a reference document corresponding to the target question, wherein the target question-answering model is trained using the question-answering model training method according to any one of claims 1 to 6.

8. A question-answering model training device, characterized in that: include: A first acquisition module, used to acquire a first question-answering dataset; The first question-answering dataset includes more than two specific enhanced samples, and the specific enhanced samples include: a specific question and an answer with a reference document corresponding to the specific question; An enhanced training module is used to use the specific question in the specific enhanced sample and the corresponding answer with the reference document as a model input sample and a model output sample, respectively, to perform enhanced training on the basic question and answer model to obtain a target question and answer model; the basic question and answer model is trained based on a second question and answer data set, and the second question and answer data set includes multiple questions and answers corresponding to each of the questions.

9. A question-answering device, characterized in that: include: The second acquisition module is used to acquire the target question; A generation module is used to input the target question into a target question-answering model to obtain an answer with a reference document corresponding to the target question, wherein the target question-answering model is trained using the question-answering model training method as described in any one of claims 1-6.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the question-answering model training method as described in any one of claims 1 to 6 is implemented; or, when the processor executes the computer program, the question-answering method as described in claim 7 is implemented.

Images