A method, system, equipment, and medium for constructing a large-scale emergency question-and-answer model.

Abstract

This invention relates to the field of artificial intelligence technology and discloses a method, system, device, and medium for constructing a large-scale emergency question-and-answer model. The method involves collecting and classifying a large amount of relevant corpus data in the emergency domain to construct an emergency corpus database. A generative large language model is then used to segment the emergency corpus database into short sentences. The generative large language model is used to improve the quality of multiple seed corpus data obtained from the sentence segmentation. Based on the high-quality fine-tuned corpus data obtained from the quality improvement process, the generative large language model is trained using zero-order optimization to obtain an emergency-specific large language model. This enables the rapid and accurate generation of practical emergency knowledge and methods for responding to emergencies, improving the response efficiency of emergency events and thus enhancing the adaptability and performance in the emergency domain.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, system, device, and medium for constructing a large-scale emergency question-and-answer model. Background Technology

[0002] In today's rapidly developing information age, emergency science popularization has become an important part of social management. Emergency science popularization covers various types of emergencies, including natural disasters, public health incidents, and sudden accidents, and faces challenges such as fragmented data, incomplete knowledge, and insufficient response timeliness. How to quickly and accurately obtain information and respond has become a major challenge for society.

[0003] With the rapid advancement of big data and artificial intelligence technologies, emergency response knowledge is highly specialized, with varying standards for classification at different levels. Managers face complex tasks and increasingly customized needs, leading to the growing vulnerability of traditional emergency response mechanisms to inflexibility and poor adaptability. New technologies are urgently needed to improve these mechanisms. In recent years, the development of generative models has brought new possibilities to the emergency response field, particularly demonstrating significant potential in Natural Language Processing (NLP).

[0004] Generative large-scale models, by simulating the distribution of real-world data, can generate content similar to it, thus offering greater flexibility in information acquisition and generation. These models can not only understand input textual information but also generate relevant responses based on context. However, current generative large-scale models struggle with emergency scenarios, failing to quickly and accurately generate practical emergency knowledge and methods for dealing with unexpected situations, resulting in poor response efficiency to emergency events. Summary of the Invention

[0005] In view of this, the present invention provides a method, system, device and medium for constructing an emergency question-and-answer large model, which solves the technical problem that current generative large models are difficult to apply to emergency scenarios and are difficult to quickly and accurately generate practical emergency knowledge and methods for dealing with emergencies, resulting in poor response efficiency to emergency events.

[0006] The first aspect of this invention provides a method for constructing a large-scale emergency question-and-answer model, comprising:

[0007] Collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database;

[0008] The emergency corpus database was segmented into short sentences using a generative large language model to obtain multiple seed corpus data.

[0009] The generative large language model is used to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data.

[0010] The generative large language model is trained using zero-order optimization based on the high-quality fine-tuned corpus data to obtain an emergency vertical domain large language model.

[0011] Preferably, the process of collecting a large amount of relevant corpus data in the field of emergency response, classifying the relevant corpus data, and constructing an emergency corpus database includes:

[0012] We collected multiple relevant corpora in the field of emergency response by crawling public data.

[0013] For each piece of relevant corpus data, keywords are extracted from the relevant corpus data using a keyword library under a preset emergency category, resulting in multiple keywords for the relevant corpus data under the preset emergency category;

[0014] Based on the relevant corpus data and multiple keywords under the preset emergency category, determine the number of keyword categories and the number of keywords under the preset emergency category in the relevant corpus data;

[0015] The preset emergency category to which the relevant corpus data belongs is determined based on the number of keyword categories and the number of keywords under the preset emergency category in the relevant corpus data. An emergency corpus database is constructed based on the relevant corpus data and its preset emergency category.

[0016] Preferably, the emergency corpus database is segmented into short sentences using a generative large language model to obtain multiple seed corpus data, including:

[0017] Using prompt word engineering techniques to guide a generative large language model, the emergency corpus database is segmented into short sentences to obtain multiple seed corpus data.

[0018] Preferably, the step of using the generative large language model to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data includes:

[0019] Context learning is performed on the seed corpus data to generate multiple example corpus data;

[0020] Based on prompt word engineering technology, multiple example corpus data are input into the generative large language model to guide the generative large language model to generate emergency instruction fine-tuning data;

[0021] By utilizing self-instruction generation technology and context learning technology, the generative large language model is guided to expand the emergency instruction fine-tuning data to obtain emergency instruction fine-tuning expanded data, which is then used as the high-quality fine-tuning corpus data.

[0022] Preferably, the step of using the generative large language model to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data includes:

[0023] Two generative large language models are used as user-generated large language models and emergency expert-generated large language models, respectively.

[0024] Based on prompt word engineering technology, the user-generated large language model and the emergency expert-generated large language model are used to conduct multi-turn dialogues using the seed corpus data to generate emergency multi-turn dialogue data, and the emergency multi-turn dialogue data is used as the high-quality fine-tuning corpus data.

[0025] Preferably, the step of performing zero-order optimization training on the generative large language model based on the high-quality fine-tuned corpus data to obtain the emergency vertical domain large language model includes:

[0026] By mixing the high-quality fine-tuned corpus data with a preset general corpus, hybrid corpus data is generated.

[0027] Based on the preset minimum loss function, the generative large language model is subjected to zero-order optimization iterative training using the mixed corpus data. After the iteration stops, the emergency vertical domain large language model is generated.

[0028] Preferably, the step of performing zero-order optimization training on the generative large language model based on the high-quality fine-tuning corpus data to obtain the emergency vertical domain large language model further includes:

[0029] The query text pre-input into the emergency vertical domain large language model is vectorized to obtain the query text vector;

[0030] The similarity between the query text vector and multiple related corpus vectors in the emergency corpus database is calculated using cosine similarity; wherein, the related corpus vectors are obtained by vectorizing the related corpus data in the emergency corpus database.

[0031] Based on the similarity calculation results, relevant corpus vectors with the highest similarity to the query text vector are selected, and the relevant corpus data corresponding to the selected relevant corpus vectors are concatenated with the query text to obtain the concatenated query text;

[0032] The concatenated query text is retrieved using the emergency vertical domain large language model, and the corresponding response text is generated.

[0033] Secondly, the present invention also provides an emergency question-and-answer large model construction system, comprising:

[0034] The corpus construction module is used to collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database.

[0035] The corpus segmentation module is used to segment the emergency corpus database into short sentences using a generative large language model to obtain multiple seed corpus data.

[0036] The corpus quality improvement module is used to improve the quality of multiple seed corpus data using the generative large language model to generate high-quality fine-tuned corpus data.

[0037] The emergency model training module is used to perform zero-order optimization training on the generative large language model based on the high-quality fine-tuning corpus data to obtain an emergency vertical domain large language model.

[0038] Thirdly, the present invention also provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the emergency question-and-answer big model construction method as described in the first aspect.

[0039] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the emergency question-and-answer large model construction method as described in the first aspect.

[0040] As can be seen from the above technical solution, this invention collects and classifies a large amount of relevant corpus data in the emergency response field to construct an emergency corpus database. It then uses a generative large language model to segment the emergency corpus database into short sentences, further improves the quality of multiple seed corpus data obtained from the sentence segmentation using the generative large language model, and performs zero-order optimization training on the generative large language model based on the high-quality fine-tuned corpus data obtained from the quality improvement process. This results in an emergency vertical domain large language model, enabling the rapid and accurate generation of practical emergency knowledge and methods for dealing with emergencies, improving the response efficiency of emergency events, and thus enhancing the adaptability and performance in the emergency response field. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This invention provides an application environment for an emergency question-and-answer large model construction method.

[0043] Figure 2 A flowchart illustrating an emergency question-and-answer large-scale model construction method provided in this embodiment of the invention;

[0044] Figure 3 This is a schematic diagram of the structure of an emergency question-and-answer large model construction system provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] The emergency question-and-answer large model construction method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the generative large language model communicates with server 102 via a network. A data storage system stores the data that server 102 needs to process. This data storage system can be integrated onto server 102 or located in the cloud or on other network servers. Server 102 collects a large amount of relevant corpus data in the emergency response domain, classifies the relevant corpus data, and constructs an emergency corpus database. It then uses the generative large language model to segment the emergency corpus database into short sentences, obtaining multiple seed corpus data. The generative large language model is then used to improve the quality of the multiple seed corpus data, generating high-quality fine-tuned corpus data. Based on the high-quality fine-tuned corpus data, the generative large language model is trained using zero-order optimization to obtain an emergency vertical domain large language model. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0048] like Figure 2 As shown in the embodiments of this application, an emergency question-and-answer large model construction method is provided, which is applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps S1 to S4. Wherein:

[0049] Step S1: Collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database.

[0050] This involves crawling relevant emergency-related data as source knowledge and extensively collecting and classifying emergency-related corpus data from large-scale open-source corpora to construct an emergency corpus database.

[0051] Specifically, step S1 involves collecting a large amount of relevant corpus data in the emergency response field, classifying the relevant corpus data, and constructing an emergency response corpus database, including:

[0052] Step S101: Collect multiple relevant corpus data in the field of emergency response by crawling public data.

[0053] Public data sources can include publicly available government emergency guidelines and high-quality datasets from online communities. Furthermore, technologies such as web scraping can be used to crawl public data and collect multiple relevant corpora in the field of emergency response.

[0054] Step S102: For each piece of relevant corpus data, extract keywords from the relevant corpus data using a keyword library under the preset emergency category to obtain multiple keywords for the relevant corpus data under the preset emergency category.

[0055] Among them, tasks in the field of emergency response can be defined and classified. For example, emergency dialogue tasks can be divided into six categories: ["prevention", "response", "in-process handling", "post-event recovery process", "regulation and system inquiry"]. For each category, a preset emergency category can be set up, and a keyword database can be built based on the emergency professional knowledge under the preset emergency category.

[0056] Since the publicly available data covers a wide range of fields, this application uses a "multi-keyword matching" strategy to extract keywords from the relevant corpus data using a keyword library under a preset emergency category, thereby obtaining multiple keywords matched by the relevant corpus data under the preset emergency category.

[0057] Step S103: Based on the relevant corpus data and multiple keywords under the preset emergency category, determine the number of keyword categories and the number of keywords under the preset emergency category.

[0058] Step S104: Determine the preset emergency category to which the relevant corpus data belongs based on the number of keyword categories and the number of keywords under the preset emergency category. Construct an emergency corpus database based on the relevant corpus data and its preset emergency category.

[0059] Specifically, when the relevant corpus data matches more than a set threshold of emergency keywords under a certain preset emergency category, the relevant corpus data is classified into a certain preset emergency category, and an emergency corpus database is constructed through the relevant corpus data and its preset emergency category.

[0060] For example, taking the data filtering for the "prevention" task as an example, the keywords selected in this invention are {"fire", "collapse", "prevention", "inspection", "maintenance", ...}, with the keyword category threshold set to 2 and the keyword number threshold set to 3. Based on the above keywords, multi-keyword matching is performed on the publicly available corpus dataset. The relevant corpus text is "Before using electrical tools, check whether the electrical circuit meets the equipment requirements to prevent electrical fires. Regularly inspect and maintain the electrical circuit to ensure the safe and reliable use of tools. Operators must be certified and strictly follow operating procedures to avoid electrical accidents." In this relevant corpus text, three keyword categories appear: "fire", "inspection", and "maintenance", and a total of four keywords appear ("fire", two "inspection", and "maintenance"). By comparison, the keyword category threshold and keyword number threshold requirements are met. Therefore, this relevant corpus text is included in the "prevention" task category.

[0061] Step S2: Use a generative large language model to segment the emergency corpus database into short sentences to obtain multiple seed corpus data.

[0062] In this embodiment, since the relevant corpus data in the emergency corpus database in step S1 have different lengths, there may be long text corpus data, which often contain knowledge from multiple emergency fields. In order to improve the accuracy and efficiency of subsequent training, the length of the relevant corpus data is divided into short sentences.

[0063] Specifically, in this embodiment of the application, the prompt word engineering technique is used to guide the generative large language model to perform short sentence segmentation on the emergency corpus database to obtain multiple seed corpus data.

[0064] Prompt engineering involves designing input text or "prompt words" to guide generative large language models in producing specific types of output to meet practical needs. These specific types include, but are not limited to, length constraints, topic coherence, structural integrity, and contextual relevance.

[0065] For example, the relevant corpus data is as follows: (I) Classification of Explosion Hazardous Locations: Explosion hazardous locations are classified into two categories according to the physical state of explosive substances: gas explosion hazardous locations and dust explosion hazardous locations. (II) Classification of Explosion Hazardous Locations: The classification principle of explosion hazardous locations is to divide areas into different hazard levels according to the frequency, duration and degree of danger of the occurrence of explosive substances. 1. Regional Classification of Gas Explosion Hazardous Locations Locations where explosive gases, combustible vapors and air are mixed to form explosive gas mixtures are divided into three regional levels according to their degree of danger. (1) Level 0 Area (abbreviated as Zone 0, the same below) Locations where explosive gas mixtures occur continuously, frequently for short periods of time or exist for long periods of time under normal conditions. (2) Level 1 Area (abbreviated as Zone 1, the same below) Locations where explosive gas mixtures may occur under normal conditions. (3) Level 3 area (hereinafter referred to as Zone 3): Under normal circumstances, explosive gas mixtures should not occur, but only occasionally for short periods under abnormal circumstances. Note: Normal circumstances refer to the normal start-up, shutdown, normal operation and maintenance of equipment. Abnormal circumstances refer to the possibility of equipment failure or misoperation. 2. Area classification of dust explosion hazard locations. Locations where explosive dust and combustible fibers mix with air to form explosive mixtures are classified into two area classifications based on their degree of danger. (1) Level 10 area: Under normal circumstances, explosive dust or combustible fiber mixtures with air may occur continuously, frequently for short periods or exist for long periods. (2) Level 11 area: Under normal circumstances, explosive dust or combustible fiber mixtures with air should not occur, but only occasionally for short periods under abnormal circumstances.

[0066] The prompt message is set to "The following is a relevant corpus data, which contains a wealth of emergency knowledge. Based on relevance, it can be reasonably decomposed into 5-10 sub-knowledge items." The prompt message can be used to reasonably decompose the above relevant corpus data into 5-10 sub-knowledge items.

[0067] Step S3: Use a generative large language model to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data.

[0068] Understandably, given the large number of seed corpus data and the fact that most of the corpus data is not accurate, of high quality, and does not have a standardized instruction data format, it is necessary to improve the quality of the seed corpus data to obtain high-quality fine-tuning corpus data that meets the standardized format and is of high quality in order to make the training more accurate and reliable.

[0069] In some embodiments, the crawled relevant knowledge is used to generate instruction-response pairs and their task types through context learning. The number of samples for each task type is counted, and to ensure more balanced model training, instruction generalization is performed on tasks with fewer samples to obtain a batch of instruction data. The language model is then fine-tuned based on this instruction data to improve its instruction recognition and response capabilities.

[0070] Specifically, step S3 involves using a generative large language model to improve the quality of multiple seed corpus data, generating high-quality fine-tuned corpus data, including:

[0071] Step S301: Perform context learning on the seed corpus data to generate multiple example corpus data.

[0072] Among them, based on the original seed corpus data, a diverse and compliant example corpus data is generated using context learning methods.

[0073] Step S302: Based on prompt word engineering technology, input multiple example corpus data into the generative large language model to guide the generative large language model to generate emergency instruction fine-tuning data.

[0074] in,

[0075] By employing prompt word engineering techniques, multiple example corpora are input into a generative large language model, thereby guiding the model to generate accurate, high-quality, and standardized instruction data. Furthermore, to statistically analyze the data distribution of the generated instruction fine-tuning dataset, this invention also requires the model to output the task classification corresponding to the generated instruction data.

[0076] For example, the prompt is set as follows: I am performing data cleaning for a large language model that can be used for emergency management. The large language model needs to act as an expert to provide reasonable feedback on user input regarding emergency situations. I will provide you with a text and related keywords and task domains. You need to generate three instruction pairs for a given task scenario based on this text and keywords. These instruction pairs can be well applied to the training of the large language model's instruction tracing. Each instruction pair is separated by two newline characters. Note that you should retain as much of the original detail in the text as possible and do not modify or delete it yourself. #Instruction Pair Output Format:

[0077] (Instruction content)", "input":{"instruction":"(User input, temporarily empty)""output":(Answer to the instruction)"","type":"(Specific task type, should be one of [(All task types under this category))"}#Example:

[0078] #Question:

[0079] Text: (txt file);

[0080] Keywords: (txt title);

[0081] Task Domain: (Task Category);

[0082] From the perspective of professionals in the relevant field, please generate three corresponding instruction pairs. The answers should be concise and clear. The instructions should be general requirements or questions based on keywords and text. The output should be the answers to the materials and instructions.

[0083] Step S303: Using self-instruction generation technology and context learning technology, guide the generative large language model to expand the emergency instruction fine-tuning data to obtain emergency instruction fine-tuning extended data, and use the emergency instruction fine-tuning extended data as high-quality fine-tuning corpus data.

[0084] Because the collected corpus data itself exhibits an imbalanced data distribution, the instruction fine-tuning dataset generated in step S22 will also suffer from this imbalance, potentially leading to overfitting during model training. Therefore, this application utilizes self-instruction generation and context learning techniques as complementary technologies. For task data with limited data distribution, it guides a generative large language model to learn features and patterns from existing data, generating similar emergency instruction fine-tuning data. This expands the emergency instruction fine-tuning data, alleviates the imbalanced data distribution, and serves as high-quality fine-tuning corpus data.

[0085] For example, the specific prompt used by the self-instruction generation technology is set as follows: You are an expert in constructing instruction pairs and also an expert in the field of [fire emergency - information delivery notification]. Please provide 10 different instruction pairs that meet the following requirements:

[0086] - The content of the instruction pair should be related to [Fire Emergency - Information Transmission Notification];

[0087] - The instructions correspond to the provided instruction pairs and have the same format as the sample.

[0088] - The content of the instruction pair must not be copied from the sample;

[0089] - Instructions must satisfy the format {\n"instruction":"(instruction content)",\n"input":"(user input, temporarily empty)",\n"output":"(answer to the instruction, detailed, comprehensive and no less than 200 words)",\n"type":"information delivery notification"\n}';

[0090] The following are several instruction pairs for the sample:

[0091] {sample}

[0092] Please generate {x} instruction pairs, with each instruction pair separated by two newline characters.

[0093] In some embodiments, step S3, which involves using a generative large language model to improve the quality of multiple seed corpus data and generate high-quality fine-tuned corpus data, further includes:

[0094] Step S311: Use two generative large language models as the user-generated large language model and the emergency expert-generated large language model, respectively.

[0095] Step S312: Based on prompt word engineering technology, use seed corpus data to enable user-generated large language model and emergency expert-generated large language model to conduct multi-turn dialogues, generate emergency multi-turn dialogue data, and use emergency multi-turn dialogue data as high-quality fine-tuning corpus data.

[0096] In this process, two generative large language models, guided by prompt word engineering technology, play the roles of user and emergency expert, respectively. These models serve as the user-generated large language model and the emergency expert-generated large language model. The two generative large language models engage in dialogue around the seed corpus data, thereby constructing emergency multi-turn dialogue data as high-quality fine-tuning corpus data.

[0097] For example, the prompt words used in multi-turn dialogue data generation are set as follows:

[0098] Role:

[0099] Emergency Experts: Possess extensive experience in emergency management, responsible for answering user questions and providing scientific emergency advice.

[0100] User: Asks questions about emergency management and first aid knowledge, seeking help.

[0101] Please conduct the following multi-round dialogue based on the emergency knowledge outlined above.

[0102] The following is a lengthy and in-depth multi-turn dialogue (at least 10 rounds) between a user and an emergency expert in this scenario, including greetings and closing. Please try to make the user's current question related to the emergency expert's previous responses or the user's own previous questions. For example, add pronouns to the current dialogue, allow the user to correct any misunderstandings by the emergency expert, and ask further questions based on the expert's responses. Below is a sample case (JSON format data) {{Selected Case}}.

[0103] Step S4: Perform zero-order optimization training on the generative large language model based on high-quality fine-tuning corpus data to obtain the emergency vertical domain large language model.

[0104] It should be noted that, since retraining a new generative large language model is complex and inefficient, this application embodiment uses high-quality fine-tuning corpus data to perform zero-order optimization training on the generative large language model, resulting in an emergency vertical domain large language model focused on emergency scenarios. This model is then used to query and generate practical emergency knowledge and methods for dealing with emergencies, helping to improve public safety awareness and self-rescue and mutual rescue capabilities. It can achieve consultation, decision support, and emotional support question-and-answer tasks in multiple scenarios such as pre-event prevention, in-event handling, and post-event recovery, reflecting high standards in terms of safety, practicality, and standardization in emergency science popularization.

[0105] Specifically, step S4 involves zero-order optimization training of the generative large language model based on high-quality fine-tuned corpus data to obtain an emergency vertical domain large language model, including:

[0106] Step S401: Generate mixed corpus data by mixing high-quality fine-tuned corpus data with preset general corpus data.

[0107] To balance the domain specificity and generality of the model, this application embodiment adopts a mixed training method using high-quality fine-tuning corpus data and general corpus data. The mixing ratio is determined according to the following formula:

[0108]

[0109] in, This indicates the amount of emergency response corpus data. This indicates the amount of general corpus data.

[0110] For example, a general corpus (that is, a corpus that covers various domains) and high-quality fine-tuning corpus data are mixed in a 1:4 ratio to avoid the large language model trained having small biases.

[0111] Step S402: Based on the preset minimum loss function, the generative large language model is trained using zero-order optimization iteration using mixed corpus data. After the iteration stops, an emergency vertical domain large language model is generated.

[0112] The Qwen2 model was pre-trained using the Next Token Prediction (NTP) task to enhance its ability to generate specialized language and understand context in emergency situations. The preset minimum loss function is:

[0113]

[0114] In the formula, Let represent the loss function for predicting the next word, and the goal is to minimize this loss function. Represents the first in the input sequence One word, This indicates that the model is working on the next word. The predicted probability, Representing the model Dimensional parameters.

[0115] It should be noted that the generative large language model in the embodiments of this application is a general-purpose large language model, such as Qwen2, gpt, qwen, llama and other large language models.

[0116] In this embodiment, the zero-order optimization algorithm (ZOA) is used to efficiently fine-tune the parameters of the generative large language model, thereby improving its adaptability and generation capabilities in the emergency response domain. Specifically, the high-dimensional overall model weight matrix is ​​adjusted... Decomposed into two low-rank matrices and In this way, the originally huge model parameter matrix... It can be done The form of the low-rank adaptor block. and The dimension of the new weight matrix is ​​much smaller than that of the original weight matrix, thus significantly reducing the number of parameters that need to be updated and greatly reducing the computational resource requirements. In actual training, a small batch dataset obtained by randomly selecting data is used. right and Zero-order fine-tuning is performed to update the parameters in order to minimize the loss function value.

[0117] The core of the zeroth-order optimization algorithm lies in using function value differences for parameter updates, eliminating the need for explicit gradient calculation and significantly reducing computational complexity. The gradient estimation formula for parameter updates is as follows:

[0118]

[0119] In the formula, It is a Gaussian distributed random variable. Characterizes the degree of disturbance. Representing the model Dimensional parameters, Indicates that it follows a distribution Small batch datasets The obtained local loss function.

[0120] It should be noted that this application embodiment collects and classifies a large amount of relevant corpus data in the emergency response field to construct an emergency corpus database. A generative large language model is used to segment the emergency corpus database into short sentences. The generative large language model is then used to improve the quality of multiple seed corpus data obtained from the sentence segmentation. Based on the high-quality fine-tuned corpus data obtained from the quality improvement process, the generative large language model is trained using zero-order optimization to obtain an emergency vertical domain large language model. This enables the rapid and accurate generation of practical emergency knowledge and methods for dealing with emergencies, improving the response efficiency of emergency events and thus enhancing the adaptability and performance in the emergency response field.

[0121] This application's embodiments construct a refined emergency scenario knowledge question-and-answer system with traceability capabilities. This invention significantly improves the performance of emergency science popularization systems under complex scenarios and knowledge traceability requirements, providing reliable technical support for efficient emergency response and knowledge dissemination, and offering valuable reference for knowledge management and generation tasks in other vertical fields.

[0122] In some embodiments, the emergency question-and-answer large model construction method provided in this application further includes:

[0123] Step S501: Vectorize the query text that has been pre-input into the emergency vertical domain large language model to obtain the query text vector.

[0124] Among them, query text refers to the question text entered by the user into the emergency vertical domain large language model according to their needs.

[0125] Step S502: Calculate the similarity between the query text vector and multiple related corpus vectors in the emergency corpus database using cosine similarity; wherein, the related corpus vectors are obtained by vectorizing the related corpus data in the emergency corpus database.

[0126] Specifically, the semantic vectorization model `acge_text_embedding` can be used to vectorize the query text and relevant corpus data in the emergency corpus database. Transform into high-dimensional related corpus vectors Its vectorized calculation formula is:

[0127]

[0128] in, This is a semantic vectorization model that ensures that the vectors can accurately capture the semantic information in the corpus.

[0129] For the query text entered by the user Similarly, perform vectorization to obtain a vector. Then, this invention calculates relevant corpus vectors using cosine similarity. With query text vector Similarity:

[0130]

[0131] In the formula, Vectors of relevant corpus With query text vector The similarity.

[0132] Specifically, when vectorizing the corpus text, the title of the corpus text is retained and not vectorized, so that in the subsequent response process, the title (such as the network source of the corpus text, etc.) is directly output, and the response generated by the emergency vertical domain large language model and the title of the corpus data used are fed back to the user.

[0133] Step S503: Based on the similarity calculation results, select the relevant corpus vectors with the highest similarity to the query text vector, and concatenate the relevant corpus data corresponding to the selected relevant corpus vectors with the query text to obtain the concatenated query text.

[0134] Among these methods, the relevant corpus data corresponding to the selected relevant corpus vectors can be concatenated with the query text to form a complete sequence input, which is then input into the emergency vertical domain large language model. This improves the retrieval of corpus data with high vector similarity to user input as evidence, thereby enhancing the refinement and accuracy of the retrieval.

[0135] Step S504: Retrieve the concatenated query text using the emergency vertical domain large language model and generate the response text corresponding to the query text.

[0136] Specifically, by using an emergency-domain large-scale language model to retrieve concatenated query texts and generate corresponding response texts, the system achieves traceable and refined emergency scenario knowledge Q&A. This significantly improves the performance of the emergency science popularization system under complex scenarios and knowledge traceability requirements, providing reliable technical support for efficient emergency response and knowledge dissemination. Based on the above data, a generative large-scale language model is trained, imbuing the emergency-domain large-scale language model with professional knowledge in the emergency field. During user queries, the model enhances the reliability of responses through retrieval-enhanced generation methods.

[0137] For example, during the query process, if the input is "how to escape safely in case of fire", the response text is: "In case of fire, you should remain calm, immediately call 119, observe whether you can escape the fire scene, and cover your mouth and nose with a wet towel."

[0138] —The above answer is from: "Fire Protection Guidebook of XX Province".

[0139] Based on the same inventive concept, this application also provides an emergency question-and-answer large model construction system for implementing the emergency question-and-answer large model construction method mentioned above.

[0140] The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more emergency question-and-answer large model construction system embodiments provided below can be found in the limitations of the emergency question-and-answer large model construction method above, and will not be repeated here.

[0141] like Figure 3 As shown in the figure, this application provides an emergency question-and-answer large model construction system, including:

[0142] The corpus construction module 100 is used to collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database.

[0143] The corpus segmentation module 200 is used to segment short sentences in the emergency corpus database using a generative large language model to obtain multiple seed corpus data.

[0144] The corpus quality improvement module 300 is used to improve the quality of multiple seed corpus data using a generative large language model, generating high-quality fine-tuned corpus data.

[0145] The emergency model training module 400 is used to perform zero-order optimization training on the generative large language model based on high-quality fine-tuning corpus data to obtain the emergency vertical domain large language model.

[0146] In some embodiments, the corpus building module 100 is used to collect multiple relevant corpus data in the field of emergency response by crawling public data;

[0147] For each relevant data entry, keywords are extracted from the relevant data entry using a keyword library under a preset emergency category, resulting in multiple keywords for the relevant data entry under the preset emergency category.

[0148] Based on multiple keywords in the relevant corpus data under the preset emergency category, determine the number of keyword categories and the number of keywords in the relevant corpus data under the preset emergency category;

[0149] The preset emergency category to which the relevant corpus data belongs is determined based on the number of keyword categories and the number of keywords under the preset emergency category. An emergency corpus database is then constructed based on the relevant corpus data and its preset emergency category.

[0150] In some embodiments, the corpus segmentation module 200 is used to guide a generative large language model to segment short sentences in an emergency corpus database using prompt word engineering techniques to obtain multiple seed corpus data.

[0151] In some embodiments, the corpus quality improvement module 300 is used to perform context learning on seed corpus data to generate multiple example corpus data;

[0152] Based on prompt word engineering technology, multiple example corpus data are input into the generative large language model to guide the generative large language model to generate emergency instruction fine-tuning data;

[0153] By utilizing self-instruction generation technology and context learning technology, a generative large language model is guided to expand the emergency instruction fine-tuning data, resulting in emergency instruction fine-tuning expanded data, which is then used as high-quality fine-tuning corpus data.

[0154] In some embodiments, the corpus quality improvement module 300 is used to utilize two generative large language models as a user-generated large language model and an emergency expert-generated large language model, respectively.

[0155] Based on prompt word engineering technology, seed corpus data is used to enable user-generated large language model and emergency expert-generated large language model to conduct multi-turn dialogues, generating emergency multi-turn dialogue data, which is then used as high-quality fine-tuning corpus data.

[0156] In some embodiments, the emergency model training module 400 is used to generate mixed corpus data by mixing high-quality fine-tuned corpus data with a preset general corpus;

[0157] Based on the preset minimum loss function, the generative large language model is trained using zero-order optimization iteration using mixed corpus data. After the iteration stops, an emergency vertical domain large language model is generated.

[0158] In some embodiments, the system includes:

[0159] The text vector module is used to vectorize the query text that has been pre-input into the emergency vertical domain large language model to obtain query text vectors;

[0160] The similarity calculation module is used to calculate the similarity between the query text vector and multiple related corpus vectors in the emergency corpus database using cosine similarity; wherein, the related corpus vectors are obtained by vectorizing the related corpus data in the emergency corpus database;

[0161] The text concatenation module is used to filter out the relevant corpus vectors with the highest similarity to the query text vector based on the similarity calculation results, and concatenate the relevant corpus data corresponding to the filtered relevant corpus vectors with the query text to obtain the concatenated query text.

[0162] The text retrieval module is used to retrieve concatenated query texts using an emergency vertical domain large language model and generate response texts corresponding to the query texts.

[0163] like Figure 4 As shown, this application provides an electronic device. The electronic device 10 includes a memory 20 and a processor 30. The memory 20 stores a computer program. When the computer program is executed by the processor 30, the processor 30 performs the steps of the emergency question-and-answer big model construction method as described in any of the above embodiments.

[0164] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the steps of the emergency question-and-answer big data model construction method as described in any of the above embodiments.

[0165] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, electronic devices, and computer storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0166] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0167] In the several embodiments provided by this invention, it will be understood that each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0168] In the embodiments provided by this invention, it should be understood that the disclosed systems, electronic devices, computer storage media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present invention through a computer device (which may be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0172] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a large-scale emergency question-and-answer model, characterized in that, include: Collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database; The emergency corpus database was segmented into short sentences using a generative large language model to obtain multiple seed corpus data. The generative large language model is used to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data, including: Context learning is performed on the seed corpus data to generate multiple example corpus data; Based on prompt word engineering technology, multiple example corpus data are input into the generative large language model to guide the generative large language model to generate emergency instruction fine-tuning data; By utilizing self-instruction generation technology and context learning technology, the generative large language model is guided to expand the emergency instruction fine-tuning data to obtain emergency instruction fine-tuning expanded data, and the emergency instruction fine-tuning expanded data is used as the high-quality fine-tuning corpus data; Two generative large language models are used as user-generated large language models and emergency expert-generated large language models, respectively. Based on prompt word engineering technology, the seed corpus data is used to enable the user-generated large language model and the emergency expert-generated large language model to conduct multi-turn dialogues, generate emergency multi-turn dialogue data, and use the emergency multi-turn dialogue data as the high-quality fine-tuning corpus data. The generative large language model is trained using zero-order optimization based on the high-quality fine-tuned corpus data to obtain an emergency vertical domain large language model, including: By mixing the high-quality fine-tuned corpus data with a preset general corpus, hybrid corpus data is generated. Based on the preset minimum loss function, the generative large language model is subjected to zero-order optimization iterative training using the mixed corpus data. After the iteration stops, the emergency vertical domain large language model is generated.

2. The emergency question-and-answer large-scale model construction method according to claim 1, characterized in that, The process involves collecting a large amount of relevant corpus data in the emergency response field, classifying the relevant corpus data, and constructing an emergency response corpus database, including: We collected multiple relevant corpora in the field of emergency response by crawling public data. For each piece of relevant corpus data, keywords are extracted from the relevant corpus data using a keyword library under a preset emergency category, resulting in multiple keywords for the relevant corpus data under the preset emergency category; Based on the relevant corpus data and multiple keywords under the preset emergency category, determine the number of keyword categories and the number of keywords under the preset emergency category in the relevant corpus data; The preset emergency category to which the relevant corpus data belongs is determined based on the number of keyword categories and the number of keywords under the preset emergency category in the relevant corpus data. An emergency corpus database is constructed based on the relevant corpus data and its preset emergency category.

3. The emergency question-and-answer large-scale model construction method according to claim 1, characterized in that, The process involves using a generative large language model to segment the emergency corpus database into short sentences, resulting in multiple seed corpus data, including: Using prompt word engineering techniques to guide a generative large language model, the emergency corpus database is segmented into short sentences to obtain multiple seed corpus data.

4. The emergency question-and-answer large-scale model construction method according to claim 1, characterized in that, The step of performing zero-order optimization training on the generative large language model based on the high-quality fine-tuned corpus data to obtain the emergency vertical domain large language model further includes: The query text pre-input into the emergency vertical domain large language model is vectorized to obtain the query text vector; The similarity between the query text vector and multiple related corpus vectors in the emergency corpus database is calculated using cosine similarity; wherein, the related corpus vectors are obtained by vectorizing the related corpus data in the emergency corpus database. Based on the similarity calculation results, relevant corpus vectors with the highest similarity to the query text vector are selected, and the relevant corpus data corresponding to the selected relevant corpus vectors are concatenated with the query text to obtain the concatenated query text; The concatenated query text is retrieved using the emergency vertical domain large language model, and the corresponding response text is generated.

5. An emergency question-and-answer large-scale model construction system, characterized in that, include: The corpus construction module is used to collect a large amount of relevant corpus data in the field of emergency response, classify the relevant corpus data, and construct an emergency corpus database. The corpus segmentation module is used to segment the emergency corpus database into short sentences using a generative large language model to obtain multiple seed corpus data. The corpus quality improvement module is used to improve the quality of multiple seed corpus data using the generative large language model to generate high-quality fine-tuned corpus data. The generative large language model is used to improve the quality of multiple seed corpus data to generate high-quality fine-tuned corpus data, including: Context learning is performed on the seed corpus data to generate multiple example corpus data; Based on prompt word engineering technology, multiple example corpus data are input into the generative large language model to guide the generative large language model to generate emergency instruction fine-tuning data; By utilizing self-instruction generation technology and context learning technology, the generative large language model is guided to expand the emergency instruction fine-tuning data to obtain emergency instruction fine-tuning expanded data, and the emergency instruction fine-tuning expanded data is used as the high-quality fine-tuning corpus data; Two generative large language models are used as user-generated large language models and emergency expert-generated large language models, respectively. Based on prompt word engineering technology, the seed corpus data is used to enable the user-generated large language model and the emergency expert-generated large language model to conduct multi-turn dialogues, generate emergency multi-turn dialogue data, and use the emergency multi-turn dialogue data as the high-quality fine-tuning corpus data. The emergency model training module is used to perform zero-order optimization training on the generative large language model based on the high-quality fine-tuning corpus data to obtain the emergency vertical domain large language model. The generative large language model is trained using zero-order optimization based on the high-quality fine-tuned corpus data to obtain an emergency vertical domain large language model, including: By mixing the high-quality fine-tuned corpus data with a preset general corpus, hybrid corpus data is generated. Based on the preset minimum loss function, the generative large language model is subjected to zero-order optimization iterative training using the mixed corpus data. After the iteration stops, the emergency vertical domain large language model is generated.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the emergency question-and-answer big data model construction method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the emergency question-and-answer large model construction method as described in any one of claims 1-4.

Images