Model training method and device, report generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a model training method and device, a report generation method and device, equipment, a medium and a program product. The model training method comprises the steps of obtaining medical terms based on medical data; the medical data comprises one or more of medical data, disease guidelines, medical literatures and case reports; obtaining multiple pieces of first text data; each piece of first text data in the plurality of pieces of first text data comprises one or more medical terms; the first text data represents text data obtained through speech recognition; identifying error vocabularies in each piece of first text data, and correcting the error vocabularies based on medical terms obtained from the medical data to obtain second text data corresponding to each piece of first text data; based on part of second text data in the multiple pieces of second text data and a preset medical consultation report corresponding to each piece of second text data in the part of second text data, executing a first fine tuning operation on the large model; the large model is used for generating a medical consultation report.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence, and particularly relates to a model training and report generation method and device, equipment, medium and program product. BACKGROUND

[0002] At present, in the traditional scheme of generating a medical report in combination with a voice recognition technology, a general medical scene is mainly targeted, and a personal physical examination report is generated. Therefore, in terms of voice recognition and medical report generation, the general medical scene is more suitable, and the professionalism and difficulty of medical report generation are both low. SUMMARY

[0003] The present application provides a model training and report generation method, device, equipment, medium and program product.

[0004] The present application provides a model training method, which comprises the following steps:

[0005] Obtaining a medical term based on medical data; the medical data comprises one or more of medical data, disease guidelines, medical literature, and case reports;

[0006] Obtaining a plurality of first text data; each of the plurality of first text data comprises one or more medical terms; the first text data represents text data obtained through voice recognition;

[0007] Identifying an error word in each of the first text data, correcting the error word based on the medical term obtained from the medical data, and obtaining second text data corresponding to each of the first text data;

[0008] Performing a first fine-tuning operation on a large model based on part of the plurality of second text data and a preset medical consultation report corresponding to each of the part of the second text data; the large model is used to generate a medical consultation report.

[0009] In some embodiments, the large model is also used to identify an error word and correct the error word; after obtaining the medical term based on the medical data, the method further comprises: obtaining a first sample data set; the first sample data set comprises an error word and a correct medical term corresponding to the error word; performing a second fine-tuning operation on the large model based on the first sample data set; identifying an error word in each of the first text data, and correcting the error word based on the medical term obtained from the medical data, comprises: correcting the error word in each of the first text data based on the large model after performing the second fine-tuning operation; performing the first fine-tuning operation on the large model, comprises: performing the first fine-tuning operation on the large model after performing the second fine-tuning operation.

[0010] It can be seen that, based on the method given in this embodiment, the large model has the ability to correct error words in the text by performing a second fine-tuning operation. By correcting error words in the first text data obtained by voice conversion through the large model, the large model trained by the corrected text data generates a medical consultation report, which can improve the accuracy of generating a medical consultation report.

[0011] In some embodiments, after obtaining the medical terms based on the medical data, the method further comprises: dividing the medical terms obtained from the medical data according to disease categories to obtain a plurality of medical terms corresponding to each disease category in different disease categories; the first sample data set is obtained, including: obtaining training text data and error words in the training text data; based on the training text data, determining the disease category to which the error words in the training text data belong; based on the disease category to which the error words in the training text data belong and the plurality of medical terms corresponding to each disease category, obtaining one or more candidate correct medical terms corresponding to the error words in the training text data; in the candidate correct medical terms, obtaining the labeled data corresponding to the error words in the training text data; based on the training text data, the error words in the training text data, the candidate correct medical terms, and the labeled data, constructing a second sample data set; the second fine-tuning operation is performed on the large model based on the first sample data set, including: performing a second fine-tuning operation on the large model based on the second sample data set.

[0012] It can be seen that, by dividing the medical terms according to the disease categories and correcting the error words based on the disease categories, it is beneficial to improve the accuracy of error correction and obtain second text data with high accuracy. By constructing a second sample data set from the corrected data and performing a second fine-tuning operation on the large model based on the second sample data set, it is beneficial to improve the error correction ability of the large model.

[0013] In some embodiments, after the medical data is used to obtain the medical terms, the method further comprises: dividing the medical terms obtained from the medical data according to disease categories to obtain a plurality of medical terms corresponding to each disease category in different disease categories; before the error words in each first text data are identified, the method further comprises: determining the disease category corresponding to each first text data; and correcting the error words in each first text data based on the large model after the second fine-tuning operation comprises: identifying the error words in each first text data by the large model after the second fine-tuning operation; obtaining one or more candidate correct medical terms corresponding to the error words in each first text data according to the disease category; and correcting the error words in each first text data based on each first text data, the error words in each first text data, and the one or more candidate correct medical terms corresponding to the error words in each first text data by the large model after the second fine-tuning operation.

[0014] As can be seen, the large model after the second fine-tuning operation can improve the error correction capability of the error words in the first text data. The large model after fine-tuning has both error correction capability and medical consultation report generation capability, so the large model after fine-tuning provided by the embodiments has the capability of obtaining accurate medical consultation reports in a professional medical consultation environment.

[0015] In some embodiments, the first fine-tuning operation on the large model based on part of the second text data and the preset medical consultation report corresponding to each second text data in the part of the second text data comprises: obtaining the preset medical consultation report corresponding to each second text data in the part of the second text data; obtaining a reconstructed medical consultation report based on the part of the second text data and the preset medical consultation report corresponding to each second text data in the part of the second text data; the reconstructed medical consultation report satisfies a preset text format; and performing supervised fine-tuning training on the large model based on the part of the second text data and the reconstructed medical consultation report corresponding to each second text data in the part of the second text data.

[0016] As can be seen, by reconstructing the preset medical consultation report and performing supervised fine-tuning training on the large model based on the reconstructed medical consultation report, the hallucination problem of the large model can be reduced, and the quality of the medical consultation report generated by the large model can be improved.

[0017] In some embodiments, after supervising the fine-tuning training of the large model, the method further includes: processing the same third text data multiple times based on the supervised fine-tuning trained large model to obtain multiple different processing results; the third text data represents any one of the second text data not used in the supervised fine-tuning training; obtaining a division result of the multiple different processing results, the division result including a first group and a second group, wherein the similarity between the processing result in the first group and the preset processing result is greater than the similarity between the processing result in the second group and the preset processing result; and performing preference optimization on the supervised fine-tuning trained large model based on any one of the processing results in the first group and any one of the processing results in the second group.

[0018] It can be seen that by optimizing the preferences of the large model after supervised fine-tuning, the large model with optimized preferences can further output medical consultation reports that meet the preset requirements, thereby further improving the accuracy of the medical consultation reports generated by the large model.

[0019] In some embodiments, after obtaining the partitioning results of the multiple different processing results, the method further includes: obtaining the optimal processing result; the optimal processing result is determined based on a preset preference criterion among the first group and the preset processing results; the step of performing preference optimization on the large model after supervised fine-tuning training based on any one processing result in the first group and any one processing result in the second group includes: performing preference optimization on the large model after supervised fine-tuning training based on the optimal processing result and any one processing result in the second group.

[0020] As can be seen, the method presented in this embodiment can further improve the diversity of data used for preference optimization, which is beneficial to improving the preference optimization results for large models.

[0021] In some embodiments, when the similarity between the first result in the first group and the second result in the second group is greater than a threshold, the first result in the first group is deleted and the second result in the second group is deleted.

[0022] As can be seen, the method given in this embodiment can improve the difference between the data in the first group and the second group. By optimizing the large model with the data that has a large difference, the accuracy of the medical consultation report output by the large model can be further improved.

[0023] This application also provides a report generation method, the method comprising:

[0024] Acquire fourth text data; the fourth text data includes one or more medical terms; the fourth text data represents text data obtained through speech recognition;

[0025] The fourth text data is processed by a large model to obtain a medical consultation report; the large model is obtained based on the above model training method.

[0026] This application embodiment also provides a model training apparatus, the apparatus comprising:

[0027] The first acquisition module is used to acquire medical terms based on medical data; the medical data includes one or more of medical data, disease guidelines, medical literature, and case reports; acquire multiple first text data; each of the multiple first text data includes one or more medical terms; the first text data represents text data obtained through speech recognition;

[0028] The error correction module is used to identify erroneous words in each of the first text data, correct the erroneous words based on medical terminology obtained from the medical data, and obtain second text data corresponding to each of the first text data.

[0029] The training module is used to perform a first fine-tuning operation on a large model based on a portion of the second text data from multiple second text data sets, and a preset medical consultation report corresponding to each of the second text data sets; the large model is used to generate medical consultation reports.

[0030] This application embodiment also provides a report generation apparatus, the apparatus comprising:

[0031] The second acquisition module is used to acquire fourth text data; the fourth text data includes one or more medical terms; the fourth text data represents text data obtained through speech recognition.

[0032] The processing module is used to process the fourth text data through a large model to obtain a medical consultation report; the large model is obtained based on the above-mentioned model training method.

[0033] This application provides an electronic device, which includes a processor and a memory for storing computer programs capable of running on the processor; wherein,

[0034] The processor is used to run the computer program to perform any of the above-described model training methods or report generation methods.

[0035] This application provides a computer storage medium storing a computer program that, when executed by a processor, implements any of the above-described model training methods or report generation methods.

[0036] This application provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described model training methods or report generation methods.

[0037] This application provides a model training and report generation method, apparatus, device, medium, and program product. By introducing medical terms from various medical materials into the error correction mechanism, the accuracy of correcting erroneous words in text data is improved. This can be used in more professional medical consultation scenarios. By training a large model with the corrected text data, the accuracy of the large model in generating medical consultation reports is improved. Attached Figure Description

[0038] Figure 1 A flowchart illustrating a method for generating a medical examination report in a related art, as provided in this application embodiment;

[0039] Figure 2 A flowchart of a model training method provided in an embodiment of this application;

[0040] Figure 3 A flowchart of a consultation report generation method provided in this application embodiment;

[0041] Figure 4 A flowchart of a large model error correction method provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram illustrating the reconstruction of a preset medical consultation report, provided as an embodiment of this application.

[0043] Figure 6 A schematic diagram of a preference optimization method provided in an embodiment of this application;

[0044] Figure 7 A flowchart of a report generation method provided in this application embodiment;

[0045] Figure 8 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the structure of a report generation device provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0048] Remote consultations emerged against the backdrop of uneven distribution of medical resources and ever-increasing medical needs. Through remote consultations, expert resources can be better utilized, medical service efficiency can be improved, medical costs can be reduced, and the diverse needs of patients can be met.

[0049] Remote consultations typically last 20-40 minutes. After the consultation, the participating medical experts will spend time drafting a medical consultation report for the original requesting hospital to guide the patient's subsequent treatment. The core points of the medical consultation report are already reflected in the consultation process; therefore, the report mainly serves as a summary of the consultation. However, current methods for automatically generating medical consultation reports primarily rely on image recognition features combined with template filling to generate an image report, such as a computed tomography (CT) report, ultrasound report, or magnetic resonance imaging (MRI) report. Some related technologies have proposed methods for directly generating an image report based on a bimodal image-language model.

[0050] Methods for generating medical examination reports are provided in other related technologies, such as Figure 1 As shown, Figure 1 A flowchart of a method for generating a physical examination report in a related technology is shown. This method involves the patient inputting examination indicators via voice, and then using speech recognition to identify the voice data 101 of the test results to obtain text data 102. A large model 103 processes the text data 102 to obtain processed structured text data 104. By combining the structured text data 104 with medical knowledge data 105 and the patient's existing personal data 106, input data 107 for the large model 103 is constructed. The input data 107 is then input into the large model 103, which generates a medical report 108 of the individual's health indicators. This method is primarily designed for general medical scenarios, generating personal physical examination reports. However, in terms of speech recognition and report generation, it is only applicable to general medical scenarios and cannot be applied to more specialized medical scenarios.

[0051] It can be seen that the main research direction in related technologies is image-based report generation, including report generation based on image feature extraction combined with text templates, or direct report output based on cross-membrane input images. In another related technology, the method for generating medical examination reports, combined with traditional speech recognition, mainly generates text data based on speech recognition, and then uses a summarization model to compress the text to generate a report. Summarization models generally include two main paths: one is an extractive summarization system, which extracts several core sentences from the text to generate a summary. This method produces summaries with missing key points, awkward sentence stacking, and a poor overall user experience; the other is a generative summarization system, which is prone to illusion problems, resulting in reports with low practicality.

[0052] To overcome the problems existing in related technologies, embodiments of this application provide a model training and report generation method, apparatus, device, medium, and program product. The model training method and report generation method provided in this application can assist doctors in writing professional medical reports for more specialized medical scenarios, improving the efficiency of doctor consultations. The method provided in this application can be applied to medical consultation scenarios focusing on discussions of difficult and complex cases, overcoming the poor performance of consultation speech recognition and the model illusion problem caused by directly training the model based on doctor-written consultation report data.

[0053] The model training method and the medical consultation report generation method given in this application embodiment can be summarized as follows: Remote medical consultation audio is used as input to a large model, and the large model outputs a medical consultation report. The remote medical consultation audio can be transcribed into text data in real time or offline. The large model is then used to abstract and summarize the text data to generate a medical consultation report containing multiple modules such as a summary of the patient's condition, diagnostic analysis, and treatment suggestions.

[0054] The model training method presented in this embodiment, through end-to-end optimization, completes a high-quality automatic remote consultation report generation solution, capable of correcting speech recognition errors in professional scenarios. Medical consultations are highly specialized scenarios, involving extensive communication of medical terminology and a range of technical vocabulary. While these terms are relatively easy for professionals to understand, they pose significant challenges for speech recognition systems. If recognition errors are not corrected, systemic errors will occur in the generation of the entire medical consultation report. This embodiment overcomes the challenge of technical terminology recognition errors by specifically correcting speech recognition results through data and model debugging and optimization. Using the method presented in this application, the final large-scale model can generate accurate medical consultation reports.

[0055] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0056] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other related elements in the method or apparatus that includes that element (e.g., steps in the method or units in the apparatus; for example, a unit in the apparatus may be a portion of circuitry, a portion of a processor, a portion of a program or software, etc.).

[0057] The model training method and report generation method provided in this application embodiment include a series of steps. However, the model training method and report generation method provided in this application embodiment are not limited to the steps described. Similarly, the model training device and report generation device provided in this application embodiment include a series of modules. However, the device provided in this application embodiment is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0058] This application provides a model training method, such as... Figure 2 As shown, Figure 2 A flowchart of a model training method is shown. Figure 2 The model training methods shown include:

[0059] Step 201: Obtain medical terminology based on medical data; medical data includes one or more of the following: medical data, disease guidelines, medical literature, and case reports.

[0060] In this embodiment, the pre-training corpus primarily incorporates medical terms obtained from various medical sources. Specifically, these sources include, but are not limited to, medical data, healthcare data, disease guidelines, medical literature, and case reports. The medical sources used to determine the correct medical terms must be authoritative and relevant to the medical field. In practical applications, the medical data can be processed using a processing model to extract medical terms. The medical terms involved in this embodiment include, but are not limited to, Chinese medical terms, English medical terms, and medical terminology abbreviations.

[0061] Before acquiring medical terminology, various medical materials can be categorized according to the type of disease. By using multiple medical materials corresponding to each disease type after categorization, the medical terminology corresponding to different disease types can be obtained.

[0062] During the pre-training phase of the large model, the context of the medical data can be set with 8192 tokens. Specifically, the text in all the medical data pre-trained by the large model can be shuffled and then concatenated piece by piece. The text can be separated by end-of-sentence tokens (EOS tokens). After segmenting the text using a text segmentation tool called tokenizer, the identity document (ID) corresponding to each segmented text is obtained. Then, the text is divided into segments according to the 8192 tokens to obtain multiple tokens. For example, multiple tokens of size 20 bytes can be obtained.

[0063] In this step, medical terms can be obtained manually or identified through relevant processing models.

[0064] Step 202: Obtain multiple first text data; each of the multiple first text data includes one or more medical terms; the first text data represents text data obtained through speech recognition.

[0065] The first text data in this step is the text data obtained from speech recognition. Specifically, in a medical consultation scenario, speech acquisition devices can collect the speech data of doctors communicating during the consultation. For example, the collected speech data can be transcribed in real time using the Jiutian Speech Model to obtain the first text data from the medical consultation process. Alternatively, the first text data corresponding to the speech can be obtained through a smart assistant. During the fine-tuning of the large model, multiple first text data can be acquired, and the first fine-tuning operation can be performed on the large model based on these multiple first text data. In practical applications, to improve the processing capabilities of the large model, the first text data can also be text data obtained from speech conversion in other medical scenarios such as consultation scenarios.

[0066] Because medical consultations involve a large number of specialized medical terms, the initial text data obtained through speech processing will contain misidentified words, especially incorrect medical terms. Therefore, the initial text data obtained by transcribing the collected speech typically includes one or more incorrect medical terms.

[0067] Figure 3A flowchart illustrating a consultation report generation method is provided. It shows that when medical consultation speech is collected via a voice acquisition device, Automatic Speech Recognition (ASR) technology is used to process the speech into multiple text segments corresponding to the speech segments. Errors in these text segments are then identified and corrected. Further, through the following steps, a first fine-tuning operation is performed on a large model based on the corrected text data. This allows the large model, after the first fine-tuning operation, to generate a medical consultation report based on the corrected text data.

[0068] Step 203: Identify erroneous words in each first text data, correct the erroneous words based on medical terminology obtained from medical materials, and obtain the second text data corresponding to each first text data.

[0069] After obtaining the first text data corresponding to the speech collected in the medical scenario, erroneous words in each first text data are identified. For example, based on multiple correct medical terms obtained in step 201, the similarity between words in the first text data and correct medical terms can be used to determine the erroneous words in the first text data. The erroneous words are then corrected based on the correct medical terms, resulting in the corrected second text data corresponding to each first text data. Alternatively, a large model can be trained for error correction, enabling the identification and correction of erroneous words in the first text data.

[0070] Step 204: Based on a portion of the second text data from multiple second text data sets, and the preset medical consultation report corresponding to each second text data set in the portion of the second text data sets, perform the first fine-tuning operation on the large model; the large model is used to generate the medical consultation report.

[0071] The above steps yielded a large amount of first text data for the initial fine-tuning training of the large model, and also a large amount of second text data after error correction. From the obtained second text data, a subset was selected, and corresponding preset medical consultation reports were generated. These preset medical consultation reports can be written by medical experts based on the second text data, or they can be reports generated by other large models, such as GPT (Generative Pre-trained Transformer), based on the second text data, which are then modified by doctors. To mitigate the large model illusion problem, a uniform format was set for the preset medical consultation reports, resulting in a uniformly formatted preset medical consultation report for each set of second text data.

[0072] Based on a portion of the second text data and the preset medical consultation report corresponding to each second text data in the portion of the second text data, the first fine-tuning operation is performed on the large model, so that the large model after the first fine-tuning operation can obtain a medical consultation report in a preset format based on the text data obtained during the medical consultation process.

[0073] This application provides a model training method that can obtain highly specialized and accurate medical terminology based on various medical resources. Using this correct medical terminology, the method can correct errors in the first text data obtained from medical consultation audio, resulting in accurate second text data. Furthermore, a first fine-tuning operation is performed on a large model using the highly accurate second text data, enabling the large model to generate medical consultation reports. This allows the large model to be applied in highly specialized medical consultation environments, generating accurate medical consultation reports. For example, the large model involved in this application embodiment can be the large model CM-13.9B. Those skilled in the art will understand that in practical applications, other large models can also be fine-tuned based on the method provided in this application embodiment, enabling them to also recognize and correct medical terminology and generate medical consultation reports.

[0074] In practical applications, steps 201 to 204 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0075] In some embodiments, the large model is further used to identify and correct erroneous words; after obtaining medical terms based on medical data, the method further includes: obtaining a first sample dataset; the first sample dataset includes erroneous words and the correct medical terms corresponding to the erroneous words; performing a second fine-tuning operation on the large model based on the first sample dataset; identifying erroneous words in each first text data and correcting erroneous words based on medical terms obtained from medical data includes: correcting erroneous words in each first text data based on the large model after performing the second fine-tuning operation; performing a first fine-tuning operation on the large model includes: performing a first fine-tuning operation on the large model after performing the second fine-tuning operation.

[0076] Before performing the first fine-tuning operation on the large model, a second fine-tuning operation is performed first. This second fine-tuning operation is used to train the large model's ability to identify and correct erroneous words, especially professional medical terminology. In other words, the large model in this embodiment of the application simultaneously possesses the ability to identify and correct words in text and generate medical consultation reports.

[0077] First, a first sample dataset is obtained for performing the second fine-tuning operation. This dataset includes erroneous words and their corresponding correct medical terms. Erroneous words include, but are not limited to, Chinese and English medical terms, medical abbreviations, and even incorrect terms corresponding to everyday expressions. After obtaining the larger model after the second fine-tuning operation, this model can identify erroneous words in the first text data obtained from speech processing, particularly incorrect medical terms. Furthermore, it can correct the erroneous words in the first text data based on the larger model after the second fine-tuning operation, thus obtaining correct second text data.

[0078] After obtaining the second text data, based on the large model after the second fine-tuning operation, the first fine-tuning operation is performed again, so that the final fine-tuned large model can correct errors based on the first text data of speech recognition, obtain the second text data, and generate a medical consultation report based on the second text data.

[0079] In some embodiments, after obtaining medical terms based on medical data, the method further includes: classifying the medical terms obtained from the medical data according to disease types to obtain multiple medical terms corresponding to each disease type; obtaining the first sample dataset includes: obtaining training text data and erroneous words in the training text data; determining the disease type to which the erroneous words in the training text data belong based on the training text data; obtaining one or more candidate correct medical terms corresponding to the erroneous words in the training text data based on the disease type to which the erroneous words in the training text data belong and the multiple medical terms corresponding to each disease type; obtaining the annotation data corresponding to the erroneous words in the training text data from the candidate correct medical terms; constructing a second sample dataset based on the training text data, the erroneous words in the training text data, the candidate correct medical terms, and the annotation data; performing a second fine-tuning operation on the large model based on the first sample dataset includes: performing a second fine-tuning operation on the large model based on the second sample dataset.

[0080] Based on the method for performing the second fine-tuning operation given in the above embodiments, this embodiment further provides that after obtaining medical terms, the medical terms are divided according to different disease categories to obtain multiple medical terms corresponding to each disease category. For example, medical entity identification can be performed first from medical guidelines. For instance, a large number of medical entities can be identified from 10,000 medical guidelines. Medical entities can cover entities such as diseases, drugs, examinations, tests, and clinical findings. Here, medical entities can be used as medical terms. A hierarchical index is established based on disease categories, using the core disease or disease category corresponding to the medical guide as keywords and the medical entities identified in the medical guide as the content corresponding to the keywords, to establish a hierarchical guide entity index library. In practical applications, based on the patient's diagnostic information obtained during the medical consultation process, such as the type of disease the patient suffers from, the disease category can be used as keywords to recall potentially related medical guidelines in the hierarchical guide entity index library, obtaining a list of entities from all related medical guidelines. Then, based on the identified incorrect words, the pronunciation similarity is calculated one by one with the entities in the related medical guidelines, recalling one or more candidate medical entities with a pronunciation similarity greater than the pronunciation similarity threshold, thus obtaining one or more candidate correct medical terms.

[0081] In constructing the second sample dataset for the second fine-tuning operation, multiple training text datasets and labeled erroneous words within them are first acquired. Here, the training text dataset can be text data obtained through speech processing of everyday medical consultation dialogues, or text data obtained after processing medical consultation speech. The labeled erroneous words in the training text dataset can be identified by a large model or labeled with the help of doctors. Based on the semantic information of the training text dataset, or by extracting keywords from the training text dataset, the types of diseases involved in the training text dataset can be determined, i.e., the disease types corresponding to the erroneous words in the training text dataset can be identified. Based on the identified disease types as keywords, one or more candidate correct medical terms corresponding to the erroneous words are determined.

[0082] From a pool of one or more candidate correct medical terms, medical experts can identify the unique correct medical term corresponding to the incorrect term. Based on the identified correct medical term, annotated training text data is obtained. A second sample dataset is constructed using the training text data, the incorrect terms in the training text data, the candidate correct medical terms, and the annotated data. Specifically, the annotated training text data can be the annotated training text data obtained based on the identified correct medical terms. A second fine-tuning operation is then performed on the large model based on the second sample dataset.

[0083] In real-world medical consultation scenarios, highly specialized medical terminology is often involved. The results of speech recognition in medical consultations frequently contain errors in recognizing these specialized medical terms, impacting the performance of downstream large-scale models. To address this, the model error correction training method presented in this application introduces a professional medical terminology annotation team to correct the speech recognition results, generating a large amount of corrected speech recognition data—i.e., annotated training text data—used to construct specialized speech recognition error correction data for medical consultation scenarios. Furthermore, the error correction capabilities of large-scale models can be enhanced by applying various medical error correction data from non-consultation scenarios. Additionally, it enhances the recognition of medical terminology abbreviations, such as the abbreviation IPF in idiopathic pulmonary fibrosis (IPF).

[0084] Because medical terms in the medical field may have similar pronunciations, and medical consultation scenarios often involve the pronunciation of many English abbreviations, the accuracy of speech recognition error correction processing using models is relatively low. Based on the second fine-tuning method given in this embodiment, the error correction capability of large models can be improved, and accurate second text data can be obtained.

[0085] In some embodiments, after performing a second fine-tuning operation on the large model based on the above method, the large model possesses the ability to correct professional medical terms in text data. Based on the method given in the above embodiments, after obtaining medical terms from medical data, the method further includes: classifying the medical terms obtained from medical data according to disease types to obtain multiple medical terms corresponding to each disease type; before identifying erroneous words in each first text data, the method further includes: determining the disease type corresponding to each first text data; the correction of erroneous words in each first text data based on the large model after performing the second fine-tuning operation includes: identifying erroneous words in each first text data through the large model after performing the second fine-tuning operation; obtaining one or more candidate correct medical terms corresponding to erroneous words in each first text data according to the disease type; and correcting erroneous words in each first text data based on each first text data, each erroneous word in each first text data, and one or more candidate correct medical terms corresponding to erroneous words in each first text data through the large model after performing the second fine-tuning operation.

[0086] After performing the second fine-tuning operation, before step 203 identifies erroneous words in each first text data, the disease type corresponding to each first text data can first be determined based on the speech information of each first text data or the keywords in each first text data.

[0087] Based on the large model after performing the second fine-tuning operation, it can identify erroneous words in the first text data and, in step 201, obtain one or more candidate correct medical terms corresponding to each erroneous word in the first text data from the medical terms corresponding to the disease types. Based on the large model after performing the second fine-tuning operation, it can determine the correct medical term corresponding to the erroneous word in the first text data from one or more candidate correct medical terms, correct the erroneous words, and obtain the second text data.

[0088] Based on the large model error correction method given in the above embodiments, the following is an example of text data recognition and error correction in a medical consultation scenario:

[0089] Suppose the first text data obtained by processing the speech data in the medical consultation scenario is: "Role 2: Can we say that pirfenidone can be stopped now? Is that what you mean? Right. \nRole 1: Ah, then the blood sugar must be controlled well. Ah, the blood sugar must be controlled well. Monitor his blood sugar level. Whether you use insulin or whatever, anyway, the blood sugar must be controlled well. This is a very important risk factor. Um\nRole 2: Mainly, you can see through his imaging that his condition has worsened this time. Do you still consider it to be P4P? \nRole 1: The possibility is very high. \nRole 2: Right? \nRole 1: Of course, we also need to be vigilant about whether there will be a problem of AE fibrosis after a sudden fever due to a virus infection, that is, acute exacerbation of pulmonary fibrosis. Right. \nRole 2: That is IPF. \nRole 2: He didn't have diabetes before, that is, during this hospitalization."

[0090] Among them, the "\n" in the above first text data is a line break character, indicating a new line.

[0091] For the above example of the first text data, the wrong words with incorrect recognition of medical terms can be obtained through the large model after performing the second fine-tuning operation. The following format example can be constructed, that is, the error correction example is as follows:

[0092]

[0093] "You are now responsible for the text error detection task after speech recognition. Please carefully analyze the possible incorrect parts in the following medical ASR text and output them in a list format. Note that only list the possible incorrect phrases, without explanation. The text to be recognized is as follows: ```{raw}```"

[0094]

[0095] Among them, the "raw" part is filled with the above first text data. It can be seen that through the large model after performing the second fine-tuning operation, the wrong words recognized in the first text data include: "pirfenidone", "P4P", "IPF".

[0096] After identifying the wrong words, based on the method given in the above embodiment, first determine that the disease type corresponding to the first text data is "Respiratory Diseases / Lung Interstitial Fibrosis Guidelines". The multiple correct medical terms corresponding to the keyword can be determined with "Respiratory Diseases / Lung Interstitial Fibrosis Guidelines" as the keyword. Calculate the pronunciation similarity between the wrong words and the multiple correct medical terms, and recall one or more candidate correct medical terms.

[0097] Based on disease type, one or more candidate correct medical terms can be recalled from multiple medical terms corresponding to that disease type. For example, the specific incorrect term "bifenidone" can recall "pifenidone"; "p-4p" can recall PCP (Pneumocystis pneumonia). By using the above method for stratified recall, the false recall rate can be effectively reduced. For example, without first recalling guidelines for respiratory diseases / pulmonary interstitial fibrosis, "p-4p" might recall PSP (Progressive Supranuclear Palsy), leading to additional errors.

[0098] Based on the recalled candidate correct medical terms, the large model, after performing a second fine-tuning operation, corrects the erroneous words in the first text data to obtain the second text data. The following error correction example can be referenced:

[0099]

[0100] "You are now responsible for text correction after speech recognition. The following medical ASR text contains some potentially incorrectly recognized parts, including: 1. pirfenidone [suspected correct phrases include pirfenidone]; 2. p-p [suspected correct phrases include PCP]; 3. i-f [suspected correct phrases include IPF, PIF, PF-ILD]. The patient's preliminary diagnosis is: idiopathic pulmonary interstitial fibrosis. Please correct the ASR text based on the above information. Note that you should directly generate the corrected text without explanation, and do not generate irrelevant text. The ASR text to be corrected is as follows: ```{raw}```"

[0101] },

[0102] {

[0103] "role":"assistant",

[0104] "value": "Role 2: It can be said that pirfenidone can be stopped for him now. What you mean is, right, right.\nRole 1: Ah, then the blood sugar must be controlled well. Ah, ah, the blood sugar must be controlled well. Monitor his blood sugar level. Whether you use insulin or whatever, anyway, the blood sugar must be controlled well. This is a very important risk factor. Um\nRole 2: Director, you mean that seeing through his imaging, his condition worsened this time. Is it more likely to be PCP?\nRole 1: This possibility is very high.\nRole 2: Right?\nRole 1: Of course, we also have to be vigilant about that sudden high fever. Could there be a problem of AE fibrosis after a viral infection, that is, acute exacerbation of pulmonary fibrosis.\nRole 2: Yes, it's AE, IPF.\nRole 2: He didn't have diabetes before, just during this hospitalization."

[0105] }

[0107] [[ID=8}}

[0108] It can be seen that through the method given in this embodiment, the large model after performing the second fine-tuning operation can correct the wrong words in the first text data, directly output the corrected second text data, and perform the first fine-tuning operation.

[0109] Based on the method given in the above embodiment, Figure 4 shows a flowchart of a large model error correction method, as Figure 4 shown, error correction based on a large model includes:

[0110] Step 401: Obtain a text segment through ASR.

[0111] Obtain real-time text segments in a medical consultation environment through ASR. The first text data in the above embodiment can be obtained through text segment splicing. The text segments obtained through ASR can be "Is it more likely to be PCP considering the exacerbation?" and "Pulmonary fibrosis is IPF".

[0112] Step 402: Identify wrong words in the large model.

[0113] Based on the large model training method given in the above embodiment, the large model after performing the second fine-tuning training identifies the wrong words in the first text data. For example, the identified wrong words are "PCP" and "IPF".

[0114] Step 403: Recall voice-similar entities.

[0115] ​Corresponding to the above embodiments, taking the disease type as the keyword, multiple medical terms corresponding to the keyword are obtained. For example, through the hierarchical guideline entity index library in the above embodiments, multiple medical terms corresponding to the keyword can be obtained as "Diagnosis and Treatment Guidelines for Interstitial Lung Diseases: [PCP, IPF, Interstitial Lung Diseases, Idiopathic Fibrosis, RF, etc.]". After determining the keyword through the first text data, one or more candidate correct medical terms corresponding to the misspelled words in the first text data are determined as the phonetic similarity entities based on the keyword. For example, the recalled phonetic similarity entities can be: {"p four p": ["PCP", "PAP", "PIP"], "挨批 f": ["IPF", "IIF", "IMF"]}.

[0116] Step 404: Guide text correction based on the recalled entities.

[0117] The large model corrects the misspelled words based on one or more candidate correct medical terms recalled.

[0118] Step 405: Obtain the complete consultation dialogue.

[0119] After error correction, the complete medical consultation dialogue, that is, the second text data, is obtained. In another implementation, the text fragment obtained in step 401 can also be directly subjected to error correction processing, and the text fragments after error correction processing are spliced to obtain the second text data, forming a complete error-corrected medical consultation dialogue.

[0120] In some embodiments, the first fine-tuning operation on the large model based on some of the second text data in the multiple second text data and the preset medical consultation reports corresponding to each of the second text data in the some second text data includes: obtaining the preset medical consultation reports corresponding to each of the second text data in the some second text data; obtaining a medical consultation report after reconstruction of the preset medical consultation report based on the some second text data and the preset medical consultation reports corresponding to each of the second text data in the some second text data; the medical consultation report after reconstruction meets the preset text format; and performing supervised fine-tuning training on the large model based on the some second text data and the reconstructed medical consultation reports corresponding to each of the second text data in the some second text data.

[0121] In the method of training the large model by taking the partially corrected second text data as the input of the large model and the medical consultation reports obtained by medical experts based on each second text data as the output of the large model, due to the randomness of the writing of medical consultation reports by different medical experts, there may be a situation where the medical consultation reports corresponding to the same second text data are inconsistent. Therefore, directly fine-tuning the large model based on the medical consultation reports obtained by medical experts is likely to cause the problem of large model hallucinations and reduce the output accuracy of the large model.

[0122] To mitigate the illusion caused by large models, this application reconstructs preset medical consultation reports. Specifically, it adjusts the clarity, coherence, and expressive effect of the text in these reports. This reconstruction may involve adjusting sentence structure, replacing vocabulary, optimizing paragraph layout, and determining the text style. By reconstructing the preset medical consultation reports, each piece of second text data in the second text data can have its reconstructed medical consultation report conforming to a preset text format. Here, the preset medical consultation report can be a medical consultation report obtained by a medical expert based on the second text data.

[0123] In practical applications, when guiding the fine-tuning process of the large model, medical consultation reports from medical experts can be provided as a reference to indicate the focus of the medical consultation. The preset medical consultation report can be polished and reconstructed using the large model GPT-4 to obtain the reconstructed medical consultation report. After expert review, the reconstructed medical consultation report is used to perform the first fine-tuning operation on the large model.

[0124] Taking the use of the large model GPT-4 to refine and reconstruct a preset medical consultation report as an example, an example of refining and reconstructing a preset medical consultation report based on GPT-4 could be:

[0125]

[0126] "You are now playing the role of a clinical expert. Based on the gold standard consultation report, summarize the following remote consultation dialogue and reconstruct it into a consultation report. The main content should include three parts: a summary of the patient's condition, diagnostic considerations, and treatment recommendations. Note: 1) During the reconstruction process, focus on the key information in the gold standard consultation report. If the key information appears in the consultation dialogue, summarize it; if it is not reflected in the dialogue, it means the key information is missing; 2) The summary of the patient's condition should be concise and highlight the core points related to the pre-treatment recommendations, not exceeding 100 words; 3) The diagnostic considerations should highlight the diagnostic conclusions drawn from this consultation, mainly including diagnostic and differential diagnostic information. The content should be concise and highlight the core points and conclusions; 4) The treatment recommendations should be presented in points, such as 1) 2) etc.; 6) The style should be simple, clear, and organized, with a concise hierarchical structure. The gold standard report is as follows: ```{golden}```. The consultation dialogue is as follows: ```{dialogue}```"

[0127] } ]

[0129] }".

[0130] The "golden" section is filled with the doctor's actual written report, i.e., a preset medical consultation report; the "dialogue" section contains the corrected consultation dialogue, i.e., the second text data. An example of filling "golden" with the doctor's actual written report is shown below:

[0131] Summary of the condition

[0132] The patient was diagnosed with interstitial lung disease two years ago and had received treatment with pirfenidone and traditional Chinese medicine. Before admission, the patient experienced symptoms such as shortness of breath and cough upon exertion, and imaging showed progression of lung disease. After admission, the patient's anti-infective treatment was ineffective, and the patient had elevated blood cell counts, accompanied by hypokalemia, hyponatremia, and abnormal liver function.

[0133] #### Diagnostic Considerations

[0134] 1. Fibrotic interstitial lung disease, considered secondary to rheumatoid arthritis or Sjögren's syndrome.

[0135] 2. Acute exacerbation may be triggered by an upper respiratory tract infection.

[0136] #### Treatment Recommendations

[0137] 1. Hormone therapy: Continue using XX, and gradually reduce the dosage after the condition stabilizes.

[0138] 2. Diuretic and supportive treatment: Pay attention to electrolyte balance and use XX as appropriate.

[0139] 3. Anti-infective treatment: Adjust the antibiotic regimen, discontinue XXX, and use XX class drugs as appropriate.

[0140] 4. Respiratory support: It is recommended to continue XX oxygen therapy and avoid intubation.

[0141] 5. Immune regulation: Consider using XX drug 5g / day for N consecutive days.

[0142] 6. Transplantation assessment: If the condition does not improve, consider the feasibility of XXX.

[0143] The above data, when requested via the OpenAI GPT4 interface over the network, can generate a candidate consultation report. An example is shown below:

[0144] Summary of the condition

[0145] The patient was diagnosed with interstitial lung disease two years ago and had received treatment with pirfenidone and traditional Chinese medicine. Before admission, the patient experienced symptoms such as shortness of breath and cough upon exertion, and imaging showed progression of lung disease. After admission, the patient's anti-infective treatment was ineffective, and the patient had elevated blood cell counts, accompanied by hypokalemia, hyponatremia, and abnormal liver function.

[0146] Diagnostic considerations

[0147] 1. Fibrotic interstitial lung disease, considered secondary to rheumatoid arthritis or Sjögren's syndrome.

[0148] 2. Acute exacerbation may be triggered by an upper respiratory tract infection.

[0149] Medical recommendations

[0150] 1. Hormone therapy: Continue using XX, and gradually reduce the dosage after the condition stabilizes.

[0151] 2. Diuretic and supportive treatment: Pay attention to electrolyte balance and use XX as appropriate.

[0152] 3. Anti-infective treatment: Adjust the antibiotic regimen, discontinue XXX, and use XX class drugs as appropriate.

[0153] 4. Respiratory support: It is recommended to continue XX oxygen therapy and avoid intubation.

[0154] 5. Immune regulation: Consider using XX drug 5g / day for N consecutive days.

[0155] 6. Transplantation assessment: If the condition does not improve, consider the feasibility of XXX.

[0156] Based on the above method, a portion of the second text data and the reconstructed medical consultation report corresponding to each text data in the portion of the second text data are obtained, and the large model is trained by supervised fine-tuning (SFT).

[0157] For example, the second text data used for SFT training can be converted into the following format:

[0158]

[0159] Here, instruction is the user command, such as text correction, report generation, etc., and response is the expected standard answer. The training style adopts the chatml format.

[0160] During SFT training, the loss function for training the autoregressive loss generative model can be calculated using formula (1):

[0161]

[0162] Where N represents the number of training samples, T i y represents the length of sequence i. i,t y represents the true label at position t in sequence i. i<t P(y) represents the portion of sequence i before position t. i,t |y i<t ) indicates that in the given context yi<t In the case of predicting the next label y i,t The probability. Here, the loss function trained in the SFT stage only retains the loss function of the response part.

[0163] Based on the method for reconstructing a preset medical consultation report given in this embodiment, and the second text data obtained in the above embodiments, Figure 5 This diagram illustrates a method for reconstructing a pre-set medical consultation report. The corrected consultation dialogue 501 (the second text data) and the expert-written consultation report 502 (the pre-set medical consultation report) are input into a large-scale reconstruction model 503. Specifically, the large-scale reconstruction model 503 can be a GPT-4 model. The reconstructed consultation report 504 is obtained through the large-scale reconstruction model 503. After obtaining the reconstructed consultation report 504, it can be reviewed by medical experts to obtain the final reconstructed medical consultation report.

[0164] Since the original expert-written consultation report 502 is directly used to train the large model, it can cause the large model to suffer from hallucinations. However, the original expert-written consultation report 502 contains key clinical points, which is of great significance for guiding the large model to extract key points from medical consultation speech. Therefore, based on the reconstruction strategy given in the embodiments of this application, an effective SFT data construction scheme for generating medical consultation reports is presented, which ensures that the large model can correctly generate the key points of medical consultation reports.

[0165] In some embodiments, after supervising fine-tuning training of the large model, the method further includes: processing the same third text data multiple times based on the supervised fine-tuning trained large model to obtain multiple different processing results; the third text data represents any one of the second text data not used in the supervised fine-tuning training; obtaining the partitioning results of the multiple different processing results, the partitioning results including a first group and a second group, wherein the similarity between the processing results in the first group and the preset processing results is greater than the similarity between the processing results in the second group and the preset processing results; and performing preference optimization on the supervised fine-tuning trained large model based on any one of the processing results in the first group and any one of the processing results in the second group.

[0166] The large model trained by SFT has excellent processing capabilities, but after careful testing, it was found that when the do_sample mode (a way to control the text generated by the model) is enabled, the quality of the medical consultation reports generated by the large model trained by SFT is still uneven. Sometimes it can generate high-quality reports, and sometimes the results are poor.

[0167] Therefore, after supervised fine-tuning training of a large model, that is, after performing the first fine-tuning operation on the large model, preference optimization can be performed on the large model after supervised fine-tuning training or the large model after performing the first fine-tuning operation.

[0168] In the process of optimizing the large model's preferences, firstly, based on the multiple second text data obtained in step 203, the second text data that did not participate in the model fine-tuning in the above embodiments is selected to obtain the third text data. The large model trained by SFT processes the same third text data to obtain different processing results. For example, different initialization parameters can be used on the large model trained by SFT to produce different processing results for the same input data, or the hyperparameters of the large model trained by SFT, such as the learning rate, epoch, and batch parameters, can be adjusted so that the large model obtains different outputs when processing the same data in one training session.

[0169] Taking adjusting the temperature parameter of the large model trained by SFT as an example, the temperature parameter can be set to 0.9. This allows the large model to obtain different processing results based on the same third-party text data, improving the diversity of the processing results. For the same third-party text data, a preset number of processing results generated by the large model are obtained, such as 20 results. These 20 results are then divided into a first group and a second group. The processing results in the first group are designated as the "best answer," and the processing results in the second group are designated as the "worst answer." During the grouping process, some processing results can be manually grouped, and then the grouping model can be trained to automatically group multiple processing results. Specifically, a preset processing result corresponding to a third text data can be set as the "original result," and the similarity between each processing result and the "original result" can be calculated. Processing results that meet the similarity threshold can be assigned to the first group, and processing results that do not meet the similarity threshold can be assigned to the second group. Alternatively, the similarity between each processing result and the "original result" can be sorted in descending order, and the top 50% of the sorted results can be assigned to the first group, and the bottom 50% of the sorted results can be assigned to the second group.

[0170] After dividing the model into first and second groups, we can determine which results from the large model meet the preference requirements and which do not. By selecting any result from the first group and any result from the second group, we can perform preference optimization on the large model trained by SFT. This allows the preference-optimized large model to output a processing result that meets the preset requirements, meaning it can output a medical consultation report that is close to the "original result".

[0171] In some embodiments, after obtaining the division results of multiple different processing results, the method further includes: obtaining the optimal processing result; the optimal processing result is determined based on a preset preference criterion in the first group and the preset processing results; the above-mentioned preference optimization of the large model after supervised fine-tuning training based on any one processing result in the first group and any one processing result in the second group includes: preference optimization of the large model after supervised fine-tuning training based on the optimal processing result and any one processing result in the second group.

[0172] Based on the method presented in this embodiment, the optimal processing result can be determined by comparing multiple "best answers" in the first group with the original result. Then, based on the optimal processing result and any processing result from the second group, preference optimization can be performed on the large model after supervised fine-tuning training. Specifically, the optimal processing result can be determined manually or based on the grouped model. Here, the grouped model can specifically be the GPT-4 large model.

[0173] For example, this application provides a Best-of-21 strategy. Assuming a large model trained using SFT processes the same third-party text data to obtain 20 processing results, these 20 results can be divided into 4 groups of 5 results each. GPT-4 can be used for automated evaluation, selecting the relatively "best answer" and "worst answer" from each group of 5 results, thus forming 4 groups of "best answers" and 4 groups of "worst answers." The 4 best answers are then placed together with the "original results," shuffled, and evaluated using GPT-4 to obtain the "globally best answer" as the optimal processing result. At this point, each test data set will obtain 1 globally best answer and 4 worst answers, serving as training data for preference optimization.

[0174] By grouping the 20 results and then determining the global best answer, the difference between the "best answer" and the "worst answer" can be increased during the preference learning process, thereby improving the preference optimization effect.

[0175] In some embodiments, after obtaining the division of multiple processing results, when the similarity between the first result in the first group and the second result in the second group is greater than a threshold, the first result in the first group is deleted and the second result in the second group is deleted.

[0176] In the initial stage of segmentation, professional manual review can be used to determine the true differences between the processing results in the first and second groups. If the difference between any processing result in the first group and any processing result in the second group is not significant, or if the processing result itself contains errors, these processing results will be discarded. In subsequent processing, a specialized processing model can be trained to directly judge the processing results in the first and second groups.

[0177] When dividing multiple processing results, the third-party text data can be used as a foundation to accumulate preference data from other medical tasks and preference datasets refined from open-source datasets, and then jointly train the scoring model. For example, the following sequence can be used: [Instruction][Answer A][Answer B][Which of the above answers better responds to the instruction?] to further judge the quality of Answer A and Answer B, where the instruction represents the question or task description that needs to be answered. Through the preference optimization given in this embodiment, the processing effect is significantly improved compared to the large model trained by SFT.

[0178] In some embodiments, after preference optimization based on the above embodiments, the method further includes: constructing a sixth text data based on the second text data after removing the text data used for SFT training and the third text data; and continuously optimizing the preference of the large model after preference optimization based on the sixth text data until the large model can obtain a medical consultation report that meets the preset standards.

[0179] In this embodiment, after the first preference optimization performed in the above embodiment, the processing model can automatically divide multiple processing results to obtain a third group and a fourth group. The similarity between the processing results in the third group and the preset processing results is greater than the similarity between the processing results in the fourth group and the preset processing results.

[0180] Based on any processing result from the third group and any processing result from the fourth group, the large model is optimized again for preferences.

[0181] Based on the preference optimization method given in the above embodiments, Figure 6A schematic diagram of a preference optimization method is shown, including: processing the same third text data 601 through a large model to obtain 20 processing results 602. The 20 processing results 602 are then divided, specifically based on the GPT large model, to obtain a partitioning result 603 containing four sets of processing results, where each set includes a "best answer set" and a "worst answer set". Based on the method given in the above embodiment, the GPT large model is used to evaluate multiple "best answer sets" and "original results" to obtain a globally optimal result 604. Using the globally optimal result 604 and multiple "worst answer sets" in the partitioning result 603, recombined preference data 605 is obtained, which can be represented as: [Global optimal result 604: "Worst answer set 1"; Global optimal result 604: "Worst answer set 2"; Global optimal result 604: "Worst answer set 3"; Global optimal result 604: "Worst answer set 4"]. By reviewing the reorganization preference data 605, processing results with small differences or erroneous processing results in the reorganization preference data 605 are removed, resulting in review result 606.

[0182] This embodiment utilizes a large-scale sampling method combined with the automatic evaluation mechanism of the GPT model to determine the global best answer from multiple processing results and a preset standard processing result. The global best answer 604 is further ensured to be of high quality through manual review, making the global best answer 604 significantly different from the "worst answer", thereby ensuring the feasibility of training a high-quality reward model.

[0183] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0184] Based on the model training method given in the above embodiments, this application also provides a report generation method, such as... Figure 7 As shown, Figure 7 A flowchart of a report generation method is provided, which includes:

[0185] Step 701: Obtain fourth text data; the fourth text data includes one or more medical terms; the fourth text data represents text data obtained through speech recognition.

[0186] Step 702: Process the fourth text data using a large model to obtain a medical consultation report.

[0187] The large model is trained using the model training method described above.

[0188] The large model obtained based on the model training method given in the above embodiments shows a significant improvement in performance during the first round of ORPO training, with AlpacaEval2.0 LC-winrate improving by 20 percentage points compared to the SFT model.

[0189] Based on the model training method and report generation method provided in this application, a systematic solution is introduced that includes large-scale model error correction detection, correct medical terminology recall, and medical terminology-guided error correction. This significantly improves the accuracy of the text data corresponding to medical consultation audio, providing accurate input source information for downstream medical consultation report generation. Addressing the issue of "large model illusion" that can easily occur when using original doctor-written reports to train a large model for generating medical consultation reports, this application provides a scheme for reconstructing original doctor reports. This ensures that the reconstructed report includes clinically relevant points and maintains consistency with the original consultation dialogue content, effectively reducing the "large model illusion" problem and improving the quality of generated medical consultation reports. By performing preference learning on the large model and combining it with the Best-of-21 strategy, sufficiently high-quality and significantly differentiated preference data can be quickly and efficiently obtained, laying the foundation for training a high-quality reward model.

[0190] The model training method presented in this application can address the problem of correcting errors in speech recognition in professional scenarios. Through data and model debugging and optimization schemes, it specifically corrects speech recognition results, overcoming the challenge of recognizing professional terminology errors and effectively resolving the reliance on high-quality data for downstream report generation. This application also presents a data reconstruction processing scheme for retraining large models, significantly reducing the large model illusion problem and improving the overall quality of model generation. Furthermore, this application's implementation, based on the Best-of-21 efficient strategy, significantly improves the overall optimization of the model.

[0191] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0192] Based on the model training method proposed in the foregoing embodiments, this application also provides a model training apparatus, such as... Figure 8 As shown, the model training device includes:

[0193] The first acquisition module 801 is used to acquire medical terms based on medical data; the medical data includes one or more of medical data, disease guidelines, medical literature, and case reports; acquire multiple first text data; each of the multiple first text data includes one or more medical terms; the first text data represents text data obtained through speech recognition.

[0194] The error correction module 802 is used to identify erroneous words in each first text data, correct the erroneous words based on medical terminology obtained from medical materials, and obtain the second text data corresponding to each first text data.

[0195] Training module 803 is used to perform a first fine-tuning operation on the large model based on a portion of the second text data from multiple second text data sets, and a preset medical consultation report corresponding to each second text data set in the portion of the second text data sets; the large model is used to generate medical consultation reports.

[0196] In practical applications, the first acquisition module 801, the error correction module 802, and the training module 803 can be implemented based on a processor and a communication device.

[0197] In some embodiments, the large model is further used to identify and correct erroneous words; the first acquisition module 801 is further used to acquire a first sample dataset; the first sample dataset includes erroneous words and the correct medical terms corresponding to the erroneous words; the training module 803 is further used to perform a second fine-tuning operation on the large model based on the first sample dataset; the error correction module 802 is specifically used to correct erroneous words in each first text data based on the large model after performing the second fine-tuning operation; the training module 803 is specifically used to perform a first fine-tuning operation on the large model after performing the second fine-tuning operation.

[0198] In some embodiments, the first acquisition module 801 is further configured to classify the medical terms obtained from the medical data according to disease types, and obtain multiple medical terms corresponding to each disease type in different disease types; the first acquisition module 801 is specifically configured to acquire training text data and erroneous words in the training text data; based on the training text data, determine the disease type to which the erroneous words in the training text data belong; based on the disease type to which the erroneous words in the training text data belong, and the multiple medical terms corresponding to each disease type, acquire one or more candidate correct medical terms corresponding to the erroneous words in the training text data; among the candidate correct medical terms, acquire the annotation data corresponding to the erroneous words in the training text data; based on the training text data, the erroneous words in the training text data, the candidate correct medical terms, and the annotation data, construct a second sample dataset; the training module 803 is specifically configured to perform a second fine-tuning operation on the large model based on the second sample dataset.

[0199] In some embodiments, the first acquisition module 801 is further configured to classify the medical terms acquired from the medical data according to disease types, and obtain multiple medical terms corresponding to each disease type in different disease types; determine the disease type corresponding to each first text data; the error correction module 802 is specifically configured to identify erroneous words in each first text data by using a large model after performing the second fine-tuning operation; the first acquisition module 801 is further configured to acquire one or more candidate correct medical terms corresponding to erroneous words in each first text data according to the disease type; the error correction module 802 is specifically configured to correct erroneous words in each first text data by using a large model after performing the second fine-tuning operation based on each first text data, erroneous words in each first text data, and one or more candidate correct medical terms corresponding to erroneous words in each first text data.

[0200] In some embodiments, the first acquisition module 801 is further configured to acquire a preset medical consultation report corresponding to each second text data in the partial second text data; the training module 803 is specifically configured to obtain a medical consultation report reconstructed from the preset medical consultation report based on the partial second text data and the preset medical consultation report corresponding to each second text data in the partial second text data; the reconstructed medical consultation report meets a preset text format; and supervised fine-tuning training of the large model is performed based on the partial second text data and the reconstructed medical consultation report corresponding to each second text data in the partial second text data.

[0201] In some embodiments, the training module 803 is further configured to process the same third text data multiple times based on the large model trained under supervised fine-tuning, to obtain multiple different processing results; the third text data represents any second text data that was not used in the supervised fine-tuning training; obtain the partitioning results of the multiple different processing results, the partitioning results include a first group and a second group, the similarity between the processing results in the first group and the preset processing results is greater than the similarity between the processing results in the second group and the preset processing results; and perform preference optimization on the large model trained under supervised fine-tuning based on any processing result in the first group and any processing result in the second group.

[0202] In some embodiments, the first acquisition module 801 is further configured to acquire the optimal processing result; the optimal processing result is determined based on a preset preference criterion in the first group and the preset processing result; the training module 803 is further configured to perform preference optimization on the large model after supervised fine-tuning training based on the optimal processing result and any processing result in the second group.

[0203] In some embodiments, the training module 803 is further configured to delete the first result in the first group and the second result in the second group when the similarity between the first result in the first group and the second result in the second group is greater than a threshold.

[0204] It should be noted that the descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0205] Based on the report generation method proposed in the foregoing embodiments, this application also provides a report generation apparatus, such as... Figure 9 As shown, the model training device includes:

[0206] The second acquisition module 901 is used to acquire fourth text data; the fourth text data includes one or more medical terms; the fourth text data represents text data obtained through speech recognition.

[0207] Processing module 902 is used to process the fourth text data through a large model to obtain a medical consultation report; the large model is trained based on the above model training method.

[0208] It should be noted that the descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0209] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0210] This application also provides an electronic device. Figure 10 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application, such as... Figure 10 As shown, the electronic device 100 may include:

[0211] Memory 1001 is used to store executable instructions.

[0212] The processor 1002 is used to execute executable instructions stored in the memory 1001 to implement any of the above-mentioned model training methods.

[0213] The processor 1002 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0214] The aforementioned computer-readable storage medium or memory 1001 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0215] This application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the model training methods provided in the above embodiments.

[0216] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the model training methods provided in the above embodiments.

[0217] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0218] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0219] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0220] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0221] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0223] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A model training method, characterized in that, The method comprises: obtaining medical terms based on medical data; the medical data comprises one or more of medical data, disease guidelines, medical literature, and case reports; obtaining a plurality of first text data; each of the plurality of first text data comprises one or more medical terms; the first text data represents text data obtained through speech recognition; identifying an error word in each of the first text data, correcting the error word based on the medical terms obtained from the medical data, to obtain second text data corresponding to each of the first text data; performing a first fine-tuning operation on a large model based on part of the plurality of second text data and a preset medical consultation report corresponding to each of the part of the plurality of second text data; the large model is used to generate a medical consultation report.

2. The method of claim 1, wherein, The large model is also used to identify and correct error words; after the medical terms are obtained based on the medical data, the method further comprises: obtaining a first sample data set; the first sample data set comprises error words and correct medical terms corresponding to the error words; performing a second fine-tuning operation on the large model based on the first sample data set; The identification of the error word in each of the first text data, the correction of the error word based on the medical terms obtained from the medical data, comprises: correcting the error word in each of the first text data based on the large model after performing the second fine-tuning operation; The first fine-tuning operation on the large model comprises: performing the first fine-tuning operation on the large model after performing the second fine-tuning operation.

3. The method of claim 2, wherein, After the medical terms are obtained based on the medical data, the method further comprises: dividing the medical terms obtained from the medical data according to disease categories to obtain a plurality of medical terms corresponding to each of the disease categories; The obtaining of the first sample data set comprises: obtaining training text data and error words in the training text data; determining the disease category to which the error words in the training text data belong based on the training text data; obtaining one or more candidate correct medical terms corresponding to the error words in the training text data based on the disease category to which the error words in the training text data belong and the plurality of medical terms corresponding to each of the disease categories; obtaining labeled data corresponding to the error words in the training text data from the candidate correct medical terms; constructing a second sample data set based on the training text data, the error words in the training text data, the candidate correct medical terms, and the labeled data; The second fine-tuning operation on the large model based on the first sample data set comprises: performing a second fine-tuning operation on the large model based on the second sample data set.

4. The method of claim 2, wherein, After the medical terms are obtained based on the medical data, the method further comprises: dividing the medical terms obtained from the medical data according to disease categories to obtain a plurality of medical terms corresponding to each of the disease categories; Before the identifying the error words in each of the first text data, the method further comprises: determining a disease category corresponding to each of the first text data; the error words in each of the first text data based on the large model after performing the second fine-tuning operation, comprising: identifying the error words in each of the first text data by the large model after performing the second fine-tuning operation; According to the disease category, one or more candidate correct medical terms corresponding to the error words in each of the first text data are obtained; Based on the each of the first text data, the error words in each of the first text data, and the one or more candidate correct medical terms corresponding to the error words in each of the first text data, the error words in each of the first text data are corrected by the large model after performing the second fine-tuning operation.

5. The method of claim 1, wherein, Based on part of the second text data and the preset medical consultation report corresponding to each of the second text data, the method comprises: obtaining the preset medical consultation report corresponding to each of the second text data in the part of the second text data; Based on the part of the second text data and the preset medical consultation report corresponding to each of the second text data, the medical consultation report after reconstruction of the preset medical consultation report is obtained; the reconstructed medical consultation report meets the preset text format; Based on the part of the second text data and the reconstructed medical consultation report corresponding to each of the second text data, the large model is supervised and fine-tuned.

6. The method of claim 5, wherein, After the supervised fine-tuning of the large model, the method further comprises: based on the large model after supervised fine-tuning, processing the same third text data multiple times to obtain multiple different processing results; the third text data represents any one of the second text data not used in the supervised fine-tuning; obtaining a division result of the multiple different processing results, the division result comprising a first group and a second group, the similarity between the processing results in the first group and the preset processing result being greater than the similarity between the processing results in the second group and the preset processing result; based on any one of the processing results in the first group and any one of the processing results in the second group, the large model after the supervised fine-tuning is preferentially optimized.

7. The method of claim 6, wherein, After the division result of the multiple different processing results is obtained, the method further comprises: obtaining an optimal processing result; the optimal processing result is determined based on a preset preference standard in the first group and a preset processing result; based on any one of the processing results in the first group and any one of the processing results in the second group, the large model after the supervised fine-tuning is preferentially optimized, comprising: based on the optimal processing result and any one of the processing results in the second group, the large model after the supervised fine-tuning is preferentially optimized.

8. The method of claim 6, wherein, The method further comprises: When a similarity between the first result in the first group and the second result in the second group is greater than a threshold, the first result in the first group is deleted, and the second result in the second group is deleted.

9. A report generation method characterized by, The method comprises: obtaining fourth text data; the fourth text data comprises one or more medical terms; the fourth text data represents text data obtained through speech recognition; processing the fourth text data through a large model to obtain a medical consultation report; the large model is obtained based on the model training method of any one of claims 1 to 8.

10. A model training apparatus, comprising: The device comprises: a first obtaining module configured to obtain medical terms based on medical materials; the medical materials comprise one or more of medical data, disease guidelines, medical literature, and case reports; and obtain a plurality of first text data; each of the plurality of first text data comprises one or more medical terms; the first text data represents text data obtained through speech recognition; a correction module configured to identify an error word in each of the first text data, correct the error word based on the medical terms obtained from the medical materials, and obtain second text data corresponding to each of the first text data; a training module configured to perform a first fine-tuning operation on a large model based on part of the plurality of second text data and a preset medical consultation report corresponding to each of the part of the second text data; the large model is used to generate a medical consultation report.

11. A report generating apparatus characterized by comprising: The device comprises: a second obtaining module configured to obtain fourth text data; the fourth text data comprises one or more medical terms; the fourth text data represents text data obtained through speech recognition; a processing module configured to process the fourth text data through a large model to obtain a medical consultation report; the large model is obtained based on the model training method of any one of claims 1 to 8.

12. An electronic device, comprising: The electronic device comprises a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is configured to run the computer program to perform the method of any one of claims 1 to 8, or claim 9.

13. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method of any one of claims 1 to 8, or claim 9.

14. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 8, or claim 9. The computer program is executed by the processor to implement the method of any one of claims 1 to 8, or claim 9.

Images