A method, system, device and medium for automatically tuning a large model-based medical text structuring model that enables traceability of an extraction process
By combining group sampling and prompt word templates with a large model, and adjusting parameters based on source text fragments, the automatic optimization of the medical text structure model was achieved. This solved the problems of high cost, black box nature, and manual dependence in existing technologies, and improved the transparency and efficiency of the extraction process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTH CHINA DIGITAL HEALTH TECHNOLOGY CO LTD
- Filing Date
- 2026-01-16
- Publication Date
- 2026-06-12
AI Technical Summary
Existing methods for extracting structured medical texts suffer from high deployment costs for large models, a black-box extraction process that is difficult to trace, and traditional model tuning relies on manual processes that are costly and time-consuming, resulting in insufficient generalization ability of the models.
By constructing prompt word templates through group sampling, extracting entity and relationship information using a large model, tracing entity locations back based on source text fragments, adjusting model parameters by statistical feature frequency, and evaluating the optimal parameter combination using cross-entropy loss, the model achieves automated tuning and traceability.
It significantly improves the accuracy and efficiency of medical text structuring, reduces hardware-dependent costs, ensures the credibility and interpretability of extraction results, and reduces human intervention and subjective bias.
Smart Images
Figure CN122198178A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of smart healthcare technology, specifically to an automatic optimization method, system, device, and medium for a medical text structured model with traceable extraction process based on a large model. Background Technology
[0002] With the continuous deepening of medical informatization, medical text data has experienced explosive growth. Unstructured texts such as electronic medical records, medical literature, and clinical diagnostic reports contain a large amount of valuable medical information. How to efficiently, accurately, and interpretably extract structured information from these texts has become a key challenge in the field of medical artificial intelligence.
[0003] In existing technologies, some solutions attempt to utilize large models for structured extraction of medical text. By constructing prompt word templates to guide the model in outputting entities and relationships, the accuracy and coverage of structured extraction are improved to some extent. Furthermore, traditional rule-based text structuring models remain widely used in structured extraction of medical text due to their high computational efficiency and strong interpretability, especially when processing large-scale medical data.
[0004] However, existing technologies still have significant shortcomings in the process of extracting structured medical texts: although large models can output entities and relationships, they are highly dependent on hardware computing power, have high deployment costs, and the extraction process is black-box and lacks traceability, making it difficult to verify and interpret the extraction results; at the same time, the optimization process of traditional text structuring models relies heavily on manual annotation and rule design, and lacks an automated parameter optimization mechanism based on data distribution characteristics, resulting in insufficient model generalization ability, high optimization costs, and long cycles. Summary of the Invention
[0005] To address the technical problems of existing medical text structure extraction methods, such as high deployment costs, black-box extraction processes, and difficulty in traceability of large-model methods, as well as the reliance on manual optimization of model parameters and the lack of automation mechanisms, this application provides a method, system, device, and medium for automatic optimization of medical text structure models based on large-model extraction with traceable extraction processes. This achieves traceability of the extraction process and automation of model optimization, significantly improving the accuracy and efficiency of medical text structure processing.
[0006] Firstly, this application provides an automatic optimization method for a medical text structured model with traceable extraction process based on a large model, comprising the following steps: S1. Group and sample the raw text data in the medical field to obtain multiple sets of training data. Each set of training data corresponds to a data group and contains multiple data records. S2. Construct a prompt word template suitable for medical information extraction. The prompt word template includes task roles, medical information extraction targets, output format constraints, and requirements for source text fragments. S3. Input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record; The first structured extraction result includes entities, relationships between entities, and source text fragments corresponding to each entity; The source text fragment is the original text fragment containing the corresponding entity; S4. Based on the source text fragment, trace back the specific location of each entity in each data record, and determine the direction and degree of the relationship between entities based on the entity location; S5. For each data group, count the frequency of occurrence of each individual characteristic under each traceability feature category in all data records within that data group. The traceability feature categories include: Entities, relationships between entities, direction of relationships between entities, and degree of relationship; S6. Based on the frequency of occurrence of each feature individual under each traceability feature category, determine the priority of model parameter adjustment for each data group. The model parameters involved in the priority of model parameter adjustment correspond to the adjustable model parameters in the text structured model. Based on the priority of model parameter adjustment for each data group, a corresponding candidate model parameter combination is generated for each data group; S7. Configure the parameter combinations of each candidate model into the text structure model to generate candidate optimized text structure models corresponding to each data group, and input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model. S8. Using the first structured extraction result of each training data as labeled data, calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, and take the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination. S9. Use the optimal combination of model parameters to configure the text structuring model to obtain the optimal text structuring model, and apply the optimal text structuring model to the structuring processing of text data in the medical field.
[0007] It should be further noted that in step S1, the original text data in the medical field includes at least one of electronic medical records, medical literature, and clinical diagnostic reports.
[0008] It should be further noted that, in step S1, the basis for grouping and sampling the original text data in the medical field includes at least one of the following: Department / Section Affiliated medical institution; The city where the affiliated medical institution is located.
[0009] It should be further noted that in step S2, the prompt word template also includes instructions on the execution steps and output format constraints.
[0010] It should be further noted that in step S3, the entity includes at least one of the following: disease name, clinical symptoms, drug name, and examination items.
[0011] It should be further noted that in step S3, the relationship between entities includes at least one of the following: etiological relationship, treatment relationship, examination relationship, and symptom association relationship.
[0012] It should be further explained that in step S4, the direction of the relationship between entities is determined as follows: In a set of relationships between entities, the first entity to appear in the corresponding data record is the starting point of the relationship, and the last entity to appear is the ending point of the relationship. The direction of the relationship is from the starting point to the ending point. The degree of relationship between entities is determined as follows: The relationship between entities within the same statement is defined as a 1-degree relationship, and the relationship between entities across n statements is defined as an n+1-degree relationship.
[0013] It should be further explained that in step S5, each entity, each type of relationship between entities, each type of relationship direction, and each type of relationship degree are treated as individual features. The method for calculating the frequency of occurrence of each individual feature under each traceability feature category in all data records within a data group is as follows: count the number of times each entity, each relationship between entities, each relationship direction, and each relationship degree appears in all data records of the corresponding data group as the frequency of occurrence of that individual feature.
[0014] It should be further explained that in step S6, the method for determining the priority of model parameter adjustment for each data group based on the occurrence frequency of each feature individual under each traceability feature category is as follows: under a traceability feature category, the higher the occurrence frequency of a feature individual, the higher its corresponding adjustment priority.
[0015] It should be further noted that in step S6, the model parameters include at least one of rule weights, confidence thresholds, and rule triggering order.
[0016] It should be further explained that in step S6, the rule for generating corresponding candidate model parameter combinations for each data group based on the priority of model parameter adjustment for each data group is as follows: The adjustable model parameters in the text structure model are divided into four categories of parameters corresponding to the source feature categories; For each type of parameter, its parameter value is associated with the frequency of occurrence of a specific individual in the corresponding source traceability feature through a predefined functional relationship; Based on the priority of model parameter adjustment for the current data group, all feature individuals of the source features are traversed in descending order of priority, and the corresponding candidate model parameter values are calculated based on the frequency of occurrence of each feature individual through the corresponding functional relationship. Summarize all candidate model parameter values under the current data group to obtain the candidate model parameter combination corresponding to that data group.
[0017] It should be further noted that in step S3, the large model is a pre-trained language model based on the Transformer architecture. In step S6, the text structuring model is a rule-based engine-based structured extraction model.
[0018] It should be further noted that in step S8, the cross-entropy loss is calculated in batches, based on the overall calculation of the difference between the second structured extraction result and the corresponding first structured extraction result of all data records.
[0019] It should be further explained that in step S8, when selecting the optimal model parameter combination, if there are multiple candidate model parameter combinations with the same cross-entropy loss and all of them are the minimum, the candidate model parameter combination corresponding to the data group with the highest comprehensive frequency index is selected as the optimal model parameter combination. The comprehensive frequency index is the sum of the occurrence frequencies of all feature individuals in the current data group.
[0020] It should be further noted that in step S9, before applying the optimal text structuring model, a verification step is also included: calculating the accuracy and recall of the optimal text structuring model. When both accuracy and recall reach the preset threshold, the optimal text structuring model is put into practical application.
[0021] It should be further noted that in step S9, the medical text data includes at least one of electronic medical records, medical literature, and clinical diagnostic reports.
[0022] Secondly, this application provides a medical text structured model optimization system based on a large model, enabling traceable extraction processes, for implementing the aforementioned automatic optimization method for medical text structured models, including: The medical text data grouping and sampling module is used to group and sample raw text data in the medical field to obtain multiple sets of training data. The prompt word template building module is used to build prompt word templates suitable for medical information extraction; The large model extraction module is used to input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record. The relation attribute determination module is used to trace back the specific position of each entity in each data record based on the source text fragment, and determine the direction and degree of the relationship between entities based on the entity position; The traceability feature frequency statistics module is used to count the frequency of occurrence of each feature under each traceability feature category in all data records within each data group for each data group. The candidate parameter combination generation module is used to determine the priority of model parameter adjustment for each data group based on the frequency of occurrence of each feature individual under each traceability feature category, and generate corresponding candidate model parameter combinations for each data group based on the priority of model parameter adjustment. The text structure model extraction module is used to configure the parameters of each candidate model into the text structure model, generate candidate optimized text structure models corresponding to each data group, and input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model. The optimal parameter combination screening module is used to calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, using the first structured extraction result of each training data as the labeled data, and select the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination. The optimal model generation and application module is used to configure the text structure model using the optimal model parameter combination to obtain the optimal text structure model, and then apply the optimal text structure model to the structured processing of text data in the medical field.
[0023] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described automatic tuning method for a medical text structured model.
[0024] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described automatic tuning method for the medical text structure model.
[0025] As can be seen from the above technical solutions, this application has the following advantages: 1. This application makes the large model extraction process transparent and verifiable by tracing the position of entities in the training data and determining the direction and degree of relationships based on the source text fragments after the first structured extraction result is output using a large model, which greatly improves the credibility and usability of the medical information extraction results.
[0026] 2. This application obtains multiple sets of training data through group sampling and counts the frequency of occurrence of source features within each data group. Based on the frequency, the priority of model parameter adjustment is determined, realizing automated and refined tuning of text structure model parameters, greatly reducing the cost of manual intervention and significantly improving the optimization efficiency of text structure model.
[0027] 3. This application adopts a cross-entropy loss evaluation mechanism based on the first structured extraction result as the annotation benchmark. By batch calculating the loss value of all data records, the optimal model parameter combination is automatically selected, which ensures the objectivity and scientific nature of the model optimization process, avoids subjective bias, and improves the reliability of the text structured model optimization results.
[0028] 4. This application significantly reduces the dependence on high-performance hardware resources by using an optimized text structuring model instead of a large model for final application, effectively solving the problem of high deployment costs of large models, and enabling large-scale application of medical text structuring processing at a more economical cost. Attached Figure Description
[0029] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is a flowchart of an automatic optimization method for a medical text structured model with traceable extraction process based on a large model, according to one embodiment of this application.
[0031] Figure 2 This is a schematic block diagram of a medical text structured model optimization system based on a large model, which enables traceable extraction process.
[0032] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation
[0033] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0034] The automatic optimization method for the medical text structured model involved in this application will be described in detail below. Specific details such as particular system structures and technologies are presented for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.
[0035] In the automatic tuning method for medical text structured models involved in this application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or sets thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.
[0036] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.
[0037] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0038] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0039] The automatic optimization method for the medical text structured model provided in this application embodiment is executed by a computer device. Correspondingly, the medical text structured model optimization system based on a large model with traceable extraction process runs in the computer device.
[0040] Figure 1 This is a flowchart illustrating an embodiment of the automatic optimization method for a medical text structured model based on a large model, enabling traceable extraction processes. Figure 1 The executing entity can be a medical text structure model optimization system. Depending on different needs, the order of steps in this flowchart can be changed, and some can be omitted.
[0041] like Figure 1 As shown, the automatic tuning method for the medical text structured model based on a large model, which enables traceable extraction processes, includes: Step S1: Group and sample the original text data in the medical field to obtain multiple sets of training data. Each set of training data corresponds to a data group and contains multiple data records.
[0042] By grouping and sampling raw text data in the medical field, multiple sets of training data are obtained to ensure the representativeness of the training data in terms of departmental distribution, institution type and geographical distribution. This provides a data foundation for model optimization and avoids the impact of data bias on model performance.
[0043] In some specific embodiments, the basis for grouping and sampling raw text data in the medical field includes at least one of the following: Department / Section Affiliated medical institution; The city where the affiliated medical institution is located.
[0044] By using the department, medical institution, and city where the medical institution is located as the basis for grouping and sampling, the training data grouping is ensured to conform to the business scenarios in the medical field and have regional representativeness, thereby improving the pertinence and practicality of model optimization.
[0045] In some specific embodiments, raw text data in the medical field includes at least one of electronic medical records, medical literature, and clinical diagnostic reports.
[0046] Step S2: Construct a prompt word template suitable for medical information extraction. The prompt word template includes task roles, medical information extraction targets, output format constraints, and requirements for source text fragments.
[0047] By constructing prompt word templates that include task roles, medical information extraction objectives, output format constraints, and requirements for source text fragments, the output format and content of the large model are standardized, ensuring the integrity and traceability of the extraction results and providing support for subsequent entity location backtracking and relationship analysis.
[0048] In some specific embodiments, the prompt word template also includes instructions for execution steps and output format constraints.
[0049] By setting execution step descriptions and output format constraints in the prompt word template, the control over the output results of large models is strengthened, the standardization and consistency of the extraction results are improved, and data quality assurance is provided for automated processing.
[0050] Step S3: Input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record; The first structured extraction result includes entities, relationships between entities, and source text fragments corresponding to each entity; The source text fragment is the original text fragment containing the corresponding entity.
[0051] By inputting data records and prompt word templates into a large model to extract entity and relation information, the first structured extraction result containing entities, relations, and corresponding source text fragments is obtained, completing the preliminary structured processing of medical text and providing data input for subsequent analysis. In some specific embodiments, the entity includes at least one of the following: disease name, clinical symptom, drug name, and examination item.
[0052] By clearly defining entities as key information types in the medical field, such as disease names, clinical symptoms, drug names, and examination items, we ensure that the extracted results meet the actual needs and application value of medical scenarios.
[0053] In some specific embodiments, the relationships between entities include at least one of etiological relationships, treatment relationships, examination relationships, and symptom association relationships.
[0054] By clearly defining the relationships between entities as typical types of relationships in the medical field, such as etiological relationships, treatment relationships, examination relationships, and symptom-related relationships, the clinical significance and application value of the extracted results are enhanced.
[0055] In some specific implementations, the large model is a pre-trained language model based on the Transformer architecture.
[0056] By clearly defining that the large model is based on the Transformer architecture, the underlying foundation for technology implementation is determined, providing technical paths and architectural guidance for the implementation of the solution.
[0057] Step S4: Based on the source text fragment, trace back the specific location of each entity in each data record within that data record, and determine the direction and degree of the relationship between entities based on their locations.
[0058] By tracing the location of entities back from the source text fragments, the direction and degree of the relationship between entities are determined based on the location information, making the extraction process visible and verifiable, solving the black-box extraction problem of large models, and improving the credibility of the results.
[0059] In some specific embodiments, the direction of the relationship between entities is determined as follows: In a set of relationships between entities, the first entity to appear in the corresponding data record is the starting point of the relationship, and the last entity to appear is the ending point of the relationship. The direction of the relationship is from the starting point to the ending point. The degree of relationship between entities is determined as follows: The relationship between entities within the same statement is defined as a 1-degree relationship, and the relationship between entities across n statements is defined as an n+1-degree relationship.
[0060] By determining the direction of relationships based on the location of entities in the text and the degree of relationships based on the span of sentences, this method provides an objective and quantifiable method for describing relationships, thereby enhancing the accuracy and interpretability of relationship extraction results.
[0061] Step S5: For each data group, count the frequency of occurrence of each individual feature under each traceability feature category in all data records within that data group. The traceability feature categories include: Entities, relationships between entities, direction of relationships between entities, and degree of relationship.
[0062] By statistically analyzing the frequency of occurrence of individual features under each data group's source feature category, a feature frequency statistical system based on data distribution is established, providing data support for determining the priority of model parameter adjustment.
[0063] In some specific embodiments, each entity, each type of relationship between entities, each type of relationship direction, and each type of relationship degree are treated as a feature individual; The method for calculating the frequency of occurrence of each individual feature under each traceability feature category in all data records within a data group is as follows: count the number of times each entity, each relationship between entities, each relationship direction, and each relationship degree appears in all data records of the corresponding data group as the frequency of occurrence of that individual feature.
[0064] By using each entity, each type of relationship between entities, each relationship direction, and each relationship degree as individual features for frequency statistics, a complete feature statistics system is established to provide data support for parameter optimization.
[0065] Step S6: Based on the frequency of occurrence of each feature individual under each traceability feature category, determine the priority of model parameter adjustment for each data group. The model parameters involved in the priority of model parameter adjustment correspond to the adjustable model parameters in the text structured model. Based on the priority of model parameter adjustment for each data group, a corresponding candidate model parameter combination is generated for each data group.
[0066] By determining the priority of model parameter adjustment for data grouping based on the frequency of occurrence of characteristic individuals, candidate model parameter combinations are generated, achieving automated tuning of model parameters and reducing the cost of manual intervention and subjective errors.
[0067] In some specific embodiments, the method for determining the priority of model parameter adjustment for each data group based on the occurrence frequency of each feature individual under each traceability feature category is as follows: under a traceability feature category, the higher the occurrence frequency of a feature individual, the higher its corresponding adjustment priority.
[0068] By associating the frequency of occurrence of individual features with adjustment priority, an adjustment mechanism is established where higher frequency means higher priority, ensuring that the focus of model parameter optimization is consistent with the data distribution characteristics and improving optimization efficiency.
[0069] In some specific embodiments, the model parameters include at least one of rule weights, confidence thresholds, and rule triggering order.
[0070] By specifying the optimal text structuring model parameters as adjustable parameter types such as rule weights, confidence thresholds, and rule triggering order, specific parameter optimization objects are provided, enhancing the feasibility of the solution.
[0071] In some specific embodiments, the rule for generating corresponding candidate model parameter combinations for each data group, based on the priority of model parameter adjustment for each data group, is as follows: The adjustable model parameters in the text structure model are divided into four categories of parameters corresponding to the source feature categories; For each type of parameter, its parameter value is associated with the frequency of occurrence of a specific individual in the corresponding source traceability feature through a predefined functional relationship; Based on the priority of model parameter adjustment for the current data group, all feature individuals of the source features are traversed in descending order of priority, and the corresponding candidate model parameter values are calculated based on the frequency of occurrence of each feature individual through the corresponding functional relationship. Summarize all candidate model parameter values under the current data group to obtain the candidate model parameter combination corresponding to that data group.
[0072] By dividing the optimal text structuring model parameters into four categories corresponding to the source feature categories, and by associating the parameter values with the frequency of occurrence of individual features through predefined functional relationships, a systematic parameter optimization mechanism is established.
[0073] In some specific embodiments, the text structuring model is a rule-based engine-based structured extraction model.
[0074] By clarifying the technical architecture of the text structuring model based on the rule engine, the underlying foundation for technical implementation is determined, providing technical paths and architectural guidance for the solution implementation.
[0075] Step S7: Configure the parameter combinations of each candidate model into the text structure model to generate candidate optimized text structure models corresponding to each data group. Then, input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model.
[0076] By combining and configuring candidate model parameters into a text structured model, a candidate optimized model is generated. The second structured extraction result is obtained by inputting training data, providing comparative data for model performance evaluation and parameter selection.
[0077] Step S8: Using the first structured extraction result of each training data as labeled data, calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, and take the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination.
[0078] By using the first structured extraction result as the labeling benchmark, the cross-entropy loss of all training data in the candidate model is calculated, and the parameter combination with the minimum loss is selected as the optimal configuration, thereby achieving objective evaluation and optimization of model parameters.
[0079] In some specific embodiments, the cross-entropy loss is calculated in batches, based on the overall calculation of the difference between the second structured extraction result and the corresponding first structured extraction result of all data records.
[0080] By employing a batch calculation method to calculate the cross-entropy loss based on the overall differences of all data records, computational efficiency is improved, ensuring the comprehensiveness and accuracy of model evaluation and avoiding evaluation bias caused by fluctuations in individual data points.
[0081] In some specific embodiments, when selecting the optimal model parameter combination, if multiple candidate model parameter combinations have the same cross-entropy loss and are all the minimum, the candidate model parameter combination corresponding to the data group with the highest comprehensive frequency index is selected as the optimal model parameter combination. The comprehensive frequency index is the sum of the frequencies of all feature individuals in the current data group.
[0082] By introducing a comprehensive frequency index as an auxiliary selection criterion when the cross-entropy loss is the same, an additional decision dimension is provided for model selection, ensuring that a more representative parameter combination is selected.
[0083] Step S9: Configure the text structuring model using the optimal model parameter combination to obtain the optimal text structuring model, and apply the optimal text structuring model to the structuring processing of text data in the medical field.
[0084] By configuring a text structuring model using the optimal combination of model parameters, and applying it to text data processing in the medical field, we can achieve efficient and high-precision medical text structuring processing, thereby improving the efficiency of medical information utilization.
[0085] In some specific embodiments, a verification step is included before applying the optimal text structuring model: calculating the precision and recall of the optimal text structuring model, and when both precision and recall reach a preset threshold, the optimal text structuring model is put into practical application.
[0086] By adding accuracy and recall verification steps before model application and setting preset thresholds as quality thresholds, we can ensure that the final model meets quality requirements and improve the reliability of the solution implementation.
[0087] In some specific embodiments, medical text data includes at least one of electronic medical records, medical literature, and clinical diagnostic reports.
[0088] By clearly defining the specific types of text data in the medical field, including electronic medical records, medical literature, and clinical diagnostic reports, the scope of application and application scenarios of the solution are defined, thereby enhancing the practicality and relevance of the solution.
[0089] The following are embodiments of a medical text structured model optimization system based on a large model with traceable extraction process, provided in this application. This medical text structured model optimization system based on a large model with traceable extraction process belongs to the same inventive concept as the automatic optimization method of the medical text structured model in the above embodiments. For details not described in detail in the embodiments of the medical text structured model optimization system, please refer to the embodiments of the automatic optimization method of the medical text structured model based on a large model with traceable extraction process described above.
[0090] like Figure 2 As shown, the medical text structure model optimization system based on a large model, which enables traceable extraction processes, includes: The medical text data grouping and sampling module is used to group and sample raw text data in the medical field to obtain multiple sets of training data. The prompt word template building module is used to build prompt word templates suitable for medical information extraction; The large model extraction module is used to input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record. The relation attribute determination module is used to trace back the specific position of each entity in each data record based on the source text fragment, and determine the direction and degree of the relationship between entities based on the entity position; The traceability feature frequency statistics module is used to count the frequency of occurrence of each feature under each traceability feature category in all data records within each data group for each data group. The candidate parameter combination generation module is used to determine the priority of model parameter adjustment for each data group based on the frequency of occurrence of each feature individual under each traceability feature category, and generate corresponding candidate model parameter combinations for each data group based on the priority of model parameter adjustment. The text structure model extraction module is used to configure the parameters of each candidate model into the text structure model, generate candidate optimized text structure models corresponding to each data group, and input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model. The optimal parameter combination screening module is used to calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, using the first structured extraction result of each training data as the labeled data, and select the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination. The optimal model generation and application module is used to configure the text structure model using the optimal model parameter combination to obtain the optimal text structure model, and then apply the optimal text structure model to the structured processing of text data in the medical field.
[0091] The medical text structured model tuning system in this embodiment is used to implement an automatic tuning method for medical text structured models based on a large model, which allows for traceability of the extraction process.
[0092] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
[0093] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0094] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.
[0095] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.
[0096] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.
[0097] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."
[0098] This application also provides a storage medium storing a program product capable of implementing an automatic tuning method for a medical text structured model with traceable extraction process based on a large model. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.
[0099] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0100] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An automatic optimization method for a medical text structured model with traceable extraction process based on a large model, characterized in that, include: S1. Group and sample the raw text data in the medical field to obtain multiple sets of training data. Each set of training data corresponds to a data group and contains multiple data records. S2. Construct a prompt word template suitable for medical information extraction. The prompt word template includes task roles, medical information extraction targets, output format constraints, and requirements for source text fragments. S3. Input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record; The first structured extraction result includes entities, relationships between entities, and source text fragments corresponding to each entity; The source text fragment is the original text fragment containing the corresponding entity; S4. Based on the source text fragment, trace back the specific location of each entity in each data record, and determine the direction and degree of the relationship between entities based on the entity location; S5. For each data group, count the frequency of occurrence of each individual characteristic under each traceability feature category in all data records within that data group. The traceability feature categories include: Entities, relationships between entities, direction of relationships between entities, and degree of relationship; S6. Based on the frequency of occurrence of each feature individual under each traceability feature category, determine the priority of model parameter adjustment for each data group. The model parameters involved in the priority of model parameter adjustment correspond to the adjustable model parameters in the text structured model. Based on the priority of model parameter adjustment for each data group, a corresponding candidate model parameter combination is generated for each data group; S7. Configure the parameter combinations of each candidate model into the text structure model to generate candidate optimized text structure models corresponding to each data group, and input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model. S8. Using the first structured extraction result of each training data as labeled data, calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, and take the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination. S9. Use the optimal combination of model parameters to configure the text structuring model to obtain the optimal text structuring model, and apply the optimal text structuring model to the structuring processing of text data in the medical field.
2. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S1, the basis for grouping and sampling the original text data in the medical field includes at least one of the following: Department / Section Affiliated medical institution; The city where the affiliated medical institution is located.
3. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S4, the direction of the relationship between entities is determined as follows: In a set of relationships between entities, the first entity to appear in the corresponding data record is the starting point of the relationship, and the last entity to appear is the ending point of the relationship. The direction of the relationship is from the starting point to the ending point. The degree of relationship between entities is determined as follows: The relationship between entities within the same statement is defined as a 1-degree relationship, and the relationship between entities across n statements is defined as an n+1-degree relationship.
4. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S5, each entity, each type of relationship between entities, each type of relationship direction, and each type of relationship degree are treated as individual features. The method for calculating the frequency of occurrence of each individual feature under each traceability feature category in all data records within a data group is as follows: count the number of times each entity, each relationship between entities, each relationship direction, and each relationship degree appears in all data records of the corresponding data group as the frequency of occurrence of that individual feature.
5. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S6, based on the frequency of occurrence of each feature individual under each traceability feature category, the method for determining the priority of model parameter adjustment for each data group is as follows: under a traceability feature category, the higher the frequency of occurrence of a feature individual, the higher its corresponding adjustment priority.
6. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S6, based on the priority of model parameter adjustment for each data group, the rule for generating corresponding candidate model parameter combinations for each data group is as follows: The adjustable model parameters in the text structure model are divided into four categories of parameters corresponding to the source feature categories; For each type of parameter, its parameter value is associated with the frequency of occurrence of a specific individual in the corresponding source traceability feature through a predefined functional relationship; Based on the priority of model parameter adjustment for the current data group, all feature individuals of the source features are traversed in descending order of priority, and the corresponding candidate model parameter values are calculated based on the frequency of occurrence of each feature individual through the corresponding functional relationship. Summarize all candidate model parameter values under the current data group to obtain the candidate model parameter combination corresponding to that data group.
7. The automatic optimization method for the medical text structured model as described in claim 1, characterized in that, In step S3, the large model is a pre-trained language model based on the Transformer architecture; In step S6, the text structuring model is a rule-based engine-based structured extraction model.
8. A medical text structured model optimization system based on a large model to achieve traceable extraction process, characterized in that, An automatic tuning method for implementing a medical text structured model as described in any one of claims 1-7 includes: The medical text data grouping and sampling module is used to group and sample raw text data in the medical field to obtain multiple sets of training data. The prompt word template building module is used to build prompt word templates suitable for medical information extraction; The large model extraction module is used to input each data record and prompt word template into the large model, extract the entity and relationship information in the data record, and output the first structured extraction result of the data record. The relation attribute determination module is used to trace back the specific position of each entity in each data record based on the source text fragment, and determine the direction and degree of the relationship between entities based on the entity position; The traceability feature frequency statistics module is used to count the frequency of occurrence of each feature under each traceability feature category in all data records within each data group for each data group. The candidate parameter combination generation module is used to determine the priority of model parameter adjustment for each data group based on the frequency of occurrence of each feature individual under each traceability feature category, and generate corresponding candidate model parameter combinations for each data group based on the priority of model parameter adjustment. The text structure model extraction module is used to configure the parameters of each candidate model into the text structure model, generate candidate optimized text structure models corresponding to each data group, and input all training data into each candidate text structure model in turn, and output the second structure extraction result of each training data in each candidate text structure model. The optimal parameter combination screening module is used to calculate the cross-entropy loss between the second structured extraction result and the first structured extraction result of all training data in each candidate text structured model, using the first structured extraction result of each training data as the labeled data, and select the candidate model parameter combination corresponding to the candidate text structured model with the smallest cross-entropy loss as the optimal model parameter combination. The optimal model generation and application module is used to configure the text structure model using the optimal model parameter combination to obtain the optimal text structure model, and then apply the optimal text structure model to the structured processing of text data in the medical field.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The steps of the automatic tuning method for the medical text structured model as described in any one of claims 1-7 are implemented when the processor executes a computer program.
10. A storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the steps of the automatic tuning method for the medical text structured model as described in any one of claims 1-7.