Method and system for constructing large language model agent for personalized medical document

Abstract

The application discloses a large language model intelligent agent construction method and system for personalized medical documents, and particularly relates to the technical field of medical artificial intelligence; when processing a new case, a set of example pairs matched with the new case is selected through a preset screening strategy, that is, the utility data and clinical data of the example pairs are analyzed, the example pairs are first screened through the clinical data, candidate example pairs are determined, and then the screening evaluation index of the candidate example pairs is analyzed, so that the set of example pairs is constructed, the new case input data is filled into the placeholder of the prompt word template, the selected set of example pairs is filled into the example area, the final execution prompt word is formed, the large language model is called, the final execution prompt word is used for interaction, and the draft content of the target chapter is obtained, so that the generated draft content can better adapt to the specific situation of the new case, content rigidity caused by fixed templates is avoided, and the pertinence of the document is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical artificial intelligence, more specifically, the present application relates to a large language model agent construction method and system for personalized medical documents. BACKGROUND

[0002] Medical documents are an indispensable part of medical activities, not only as a carrier to record patient conditions, diagnosis and treatment processes, and treatment effects, but also as an important basis for medical quality control, legal responsibility definition, medical research and teaching. There are many types of common medical documents, such as admission records, first course records, preoperative discussions, surgical records, shift handover records, and discharge summaries. The writing of these documents often consumes a lot of time and effort for doctors, especially in a high-load clinical environment.

[0003] Traditional medical document writing mainly relies on manual input and editing by doctors. Some medical institutions have introduced electronic medical record systems based on fixed templates, which have improved writing efficiency to some extent, but still have the following problems:

[0004] It is not possible to construct example pairs according to high-quality anonymized medical case examples pre-labeled by target medical institutions, so that the generated content cannot meet the actual needs of the institution, increasing the workload of doctors;

[0005] In addition, when dealing with new cases, the matching degree between the new case and the constructed example pair is not analyzed from multiple dimensions, and high-quality example pairs that are more matched to the new case are not selected, so that the generated draft content cannot be more adapted to the specific circumstances of the new case, and the assistance effect of the agent is poor.

[0006] Therefore, a large language model agent construction method and system for personalized medical documents are proposed. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a large language model agent construction method and system for personalized medical documents.

[0008] To achieve the above object, the present application provides the following technical scheme:

[0009] The large language model agent construction method for personalized medical documents comprises:

[0010] Local processing: based on the target chapter generated by the large language model, the high-quality anonymized medical record cases corresponding to the target chapter are obtained, which are pre-labeled by the target medical institution. The cases contain the input information of the training agent and the reference output content of the target chapter. The input information and reference output content are extracted from the medical record cases, and the structured input-output example pairs are constructed. The input information is analyzed, the target department corresponding to the target chapter is identified, and the diagnosis and treatment term set pre-integrated by the target department is extracted as the localized input feature required for generating the target chapter;

[0011] Initial setting: based on the target chapter features, input-output example pairs and localized input features, combined with the preset prompt word engineering strategy, the initial configuration of the agent is semi-automatically generated, including the prompt word template of the dynamic data placeholder and the example area;

[0012] Case processing: when processing a new case, the initial configuration of the agent corresponding to the target chapter is selected. From the constructed input-output example pairs, the preset screening strategy is used to select and create an example pair set matched with the new case from the example pairs. The new case input data is filled into the placeholder of the prompt word template, and the selected example pair set is filled into the example area to form the final execution prompt word. The large language model is called, and the final execution prompt word is used to interact to obtain the draft content of the target chapter.

[0013] Specifically, the preset screening strategy is used to select and create an example pair set matched with the new case from the example pairs, specifically:

[0014] Clinical data is extracted from the input information of the constructed input-output example pairs, wherein the clinical data includes diagnosis information, symptom information, patient age and gender; candidate example pairs are screened from the constructed input-output example pairs based on the clinical data;

[0015] The clinical data and utility data of the candidate example pairs are analyzed and processed to construct an example pair set matched with the new case; the utility data includes historical call frequency, historical feedback score and modification character proportion.

[0016] Specifically, the candidate example pairs are screened from the constructed input-output example pairs based on the clinical data, specifically:

[0017] The age corresponding to each group of age intervals is set, and the patient age of the new case and the patient age of the example pair are converted into the corresponding age interval;

[0018] For the diagnosis information of the new case patient and the example pair patient, the diagnosis information is mapped to an ICD-10 code; if the gender, age interval and ICD-10 code of the example pair patient are consistent with the gender, age interval and ICD-10 code of the new case patient, it is determined that the matching is successful, and the example pair matched with the new case successfully is reserved as a candidate example pair.

[0019] Specifically, the analysis and processing of the clinical data of the candidate example pair are specifically:

[0020] Symptom performance word segments are extracted from the symptom information of the new case patient and the candidate example pair patient, and standardized processing is performed by using a medical terminology dictionary; after the processing is completed, a symptom word set of the new case patient and a symptom word set of the candidate example pair patient are constructed;

[0021] For the symptom word set of the new case patient and the symptom word set of the candidate example pair patient, symptom word matching is performed, the number of matched symptom words is counted as a matching number, and a proportion of the matching number in the total number of symptom words of the new case is calculated to obtain a symptom matching ratio.

[0022] Specifically, the analysis and processing of the utility data of the candidate example pair are specifically:

[0023] The number of historical invocations of the candidate example pair is counted, and each group of number intervals corresponding to the number of historical invocations is constructed in advance, and each group of number intervals corresponds to a content performance score; the number of historical invocations of the candidate example pair is matched with the corresponding number interval to determine the content performance score of the candidate example pair; the historical feedback scores after each invocation of the candidate example pair are obtained; the average value of each group of historical feedback scores of the candidate example pair is taken to obtain a content satisfaction score of the candidate example pair; the content performance score and the content satisfaction score of the candidate example pair are summed to obtain a content comprehensive score; the content comprehensive score of the candidate example pair is divided by the highest content comprehensive score to obtain a content recognition ratio; the modification character proportion of the generated content after each historical invocation of the candidate example pair is extracted, and the average value is taken as the content modification ratio of the candidate example pair.

[0024] Specifically, the construction of the example pair set matched with the new case is specifically:

[0025] The symptom matching ratio, the content recognition ratio and the content modification ratio of the candidate example pair are comprehensively processed to determine a screening evaluation index of the candidate example pair; the screening evaluation index of each candidate example pair is compared with a preset screening reference index; if the screening evaluation indices of multiple groups of candidate example pairs are higher than the screening reference index, the candidate example pairs are sorted from large to small according to the size of the screening evaluation index from left to right, and the first three groups of candidate example pairs are selected to construct an example pair set.

[0026] If the screening evaluation indexes of each candidate example pair are all lower than the preset screening reference index, the candidate example pair with a higher screening evaluation index is directly selected to construct the example pair set.

[0027] Specifically, the method further comprises:

[0028] Target identification: obtaining a preset medical document template of a target medical institution, extracting the chapter hierarchy, title and fixed text of the template, determining a target chapter generated with the assistance of the large language model agent based on the content characteristic data analysis result of the chapter; the content characteristic data includes descriptive text proportion, writing time cost and structured degree.

[0029] Specifically, the method further comprises:

[0030] Extracting the descriptive text proportion in the chapter content; combining the doctor's work record to statistically record the average writing time of each chapter; comparing the descriptive text proportion and the average writing time of each chapter with the preset proportion reference value and the writing reference time length respectively, and selecting the chapters with the descriptive text proportion and the average writing time higher than the proportion reference value and the writing reference time length as candidate chapters;

[0031] Retrieving the standardized entity description field contained in the candidate chapter, calculating the proportion of the number of standardized entity description fields in the candidate chapter to the total number of fields in the chapter to obtain the standardization rate of the candidate chapter;

[0032] The preset standardization rate corresponds to a medium structure standard rate range, and the standardization rate of the candidate chapter is matched with the preset medium structure standard rate range; if the matching is successful, the candidate chapter is determined as the target chapter generated with the assistance of the large language model agent.

[0033] The large language model agent construction system for personalized medical documents comprises:

[0034] Target identification module: obtaining a preset medical document template of a target medical institution, extracting the chapter hierarchy, title and fixed text of the template, determining a target chapter generated with the assistance of the large language model agent based on the content characteristic data analysis result of the chapter;

[0035] Case processing module: obtaining a high-quality anonymized medical record case corresponding to the target chapter and originating from the target medical institution, the case containing input information for training the agent and reference output content of the target chapter, extracting the input information and the reference output content from the medical record case, constructing a structured input-output example pair, analyzing the input information, identifying the target department corresponding to the target chapter, and extracting the pre-integrated diagnosis and treatment terminology set of the target department as the localized input features required for generating the target chapter.

[0036] Initial setting module: based on target chapter features, input-output example pairs and localized input features, combined with preset prompt word engineering strategy, semi-automatically generate initial configuration of agent, including dynamic data placeholder and prompt word template of example area;

[0037] Case processing module: when processing a new case, select the initial configuration of the agent corresponding to the target chapter, from the constructed input-output example pairs, use the preset screening strategy to select and create a set of example pairs matching the new case from the example pairs, fill the new case input data into the placeholder of the prompt word template, fill the selected example pair set into the example area, form the final execution prompt word, call the large language model, interact using the final execution prompt word, and obtain the draft content of the target chapter;

[0038] Result confirmation module: present the draft content to the user for review, editing and confirmation

[0039] Technical effects and advantages of the present application:

[0040] (1) When processing a new case, the set of example pairs matching the new case is selected by the preset screening strategy, that is, the utility data and clinical data of the example pairs are analyzed, the example pairs are first screened by the clinical data to determine the candidate example pairs, then the screening evaluation index of the candidate example pairs is analyzed, thereby constructing the set of example pairs, filling the new case input data into the placeholder of the prompt word template, filling the selected example pair set into the example area, forming the final execution prompt word, calling the large language model, interacting using the final execution prompt word, and obtaining the draft content of the target chapter, so that the generated draft content can better adapt to the specific situation of the new case, avoid content rigidity caused by fixed templates, and improve the pertinence of the document;

[0041] (2) By determining the target chapter generated by the large language model agent, focusing on chapters with high descriptive text proportion and long writing time, providing targeted assistance, reducing the manual writing time of doctors in these high-time-consuming chapters, and relieving the working pressure of doctors in high-load clinical environment, using structured input-output example pairs and preset prompt word templates, the draft content of the target chapter can be quickly generated, avoiding doctors starting from scratch, and greatly shortening the document writing period;

[0042] (3) Based on the high-quality anonymized medical record cases labeled by the target medical institution in advance, the example pairs are constructed, the diagnosis and treatment habits and document specifications of the institution are integrated, and the generated content is more in line with the actual needs of the institution, reducing the modification workload of doctors. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The flow chart of the large language model agent construction method for personalized medical document of the present application;

[0044] Figure 2 The system principle diagram for the large language model agent for personalized medical document is constructed. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0046] Embodiment 1

[0047] As shown in the figure, the large language model agent construction method for personalized medical documents comprises: Figure 1

[0048] Step a: obtaining a preset document template of a target medical institution, using natural language processing technology and document structure analysis algorithm to extract the chapter level, title and fixed text of the template, determining the target chapter assisted by the large language model agent based on the content characteristic data analysis result of the chapter or in combination with the auxiliary annotation of the user through the interactive interface; wherein the content characteristic data comprises descriptive text proportion, writing time cost and structured degree;

[0049] Collecting preset document templates (such as Word, PDF, electronic medical record system templates, etc.) of the target medical institution, uniformly formatting and converting into a parseable text format (such as JSON, XML);

[0050] Using a document structure analysis algorithm (such as LayoutLM, PDFPlumber) to extract the physical structure of the template:

[0051] Chapter level: identifying the nested relationship of “primary title-secondary title-paragraph” (such as “admission record-past history-past history”);

[0052] Fixed text and variable area: distinguishing the fixed and unchanged introductory words (such as “chief complaint:” in “chief complaint: ______”) in the template and the variable content area (such as underlined or blank) to be filled in;

[0053] Performing NLP analysis on the text content (including fixed description and historical filling examples) of each chapter to extract basic features: text length (average number of words, number of paragraphs); term density (medical professional term proportion, based on UMLS medical dictionary matching); sentence complexity (average sentence length, proportion of subordinate clauses).

[0054] Specifically:​

[0055] Extract the proportion of descriptive text in the chapter content; the descriptive content includes, but is not limited to, disease analysis and treatment strategies;

[0056] The higher the proportion of descriptive text (e.g., >60%), the better it is for generation; purely structured tables (e.g., check result items) do not require assistance;

[0057] The average writing time for each chapter was calculated by combining the doctor's work records (extracted from the operation log of the electronic medical record system).

[0058] Chapters that take longer to complete (e.g., >10 minutes) (e.g., "Summary of Medical Records") should be prioritized for inclusion in the target chapters to improve efficiency and benefits;

[0059] The proportion of descriptive text and the average writing time of each chapter are compared with the preset reference values ​​for the proportion and the reference writing time, and chapters with both the proportion of descriptive text and the average writing time being higher than the reference values ​​for the proportion and the reference writing time are selected as candidate chapters.

[0060] Retrieve the standardized entity description fields contained in the candidate chapters; the standardized entity description fields are specifically entities that have been standardized and annotated by the medical ontology, including but not limited to diseases, symptoms, and drugs; calculate the proportion of the number of standardized entity description fields in the candidate chapters to the total number of fields in the chapters, and obtain the standardization rate of the candidate chapters;

[0061] The standardization rate of candidate chapters is matched with the pre-defined range of standardization rates corresponding to the medium structure standardization rate. If the match is successful, the chapter is used as the target chapter generated by the large language model agent.

[0062] Chapters with moderate structure (with a framework but requiring flexible filling) are suitable for generation assistance; chapters with completely fixed formats (such as gender) do not need to be generated, while chapters with no structure at all (such as "special notes") are difficult to generate.

[0063] Step b: Obtain high-quality anonymized medical record cases from the target medical institution that correspond to the target chapter. The cases contain input information for training or guiding the agent and reference output content for the target chapter. Guide medical experts through the user interface to extract input information (such as chief complaint, examination results, etc.) and reference output content (text conforming to institutional standards) from the medical record cases, construct structured input-output example pairs, analyze the input information, identify the target department corresponding to the target chapter, and extract the pre-integrated set of diagnostic and treatment terms for the target department as localized input features required to generate the target chapter.

[0064] A user interaction interface is provided to guide medical professionals to map the original information (e.g. chief complaint, history of present illness, physical examination findings, key results of auxiliary examinations, common diagnosis and treatment terms and expression habits of specific departments, etc.) in the localized medical record case related to the generation of the target chapter to structured input information, and to map the text content of the corresponding target chapter in the medical record case that meets the writing specifications and quality requirements of the institution to reference output content;

[0065] High-quality historical medical record cases from the specific medical institution, recognized by senior doctors of the institution, and thoroughly anonymized, are obtained. These "localized" cases are key because they embody the best practices, common expressions, and professional standards of the institution in writing corresponding chapters of medical documents. The system provides tools or guided processes to help medical professionals or annotators accurately correspond and extract "input information" (e.g. key examination results of patients, proposed surgical procedures, major risk points, etc. that doctors usually refer to when writing a "preoperative discussion") for generating the content of the target chapter, and the "reference output content" actually written in these high-quality cases. Through this process, a set of structured "input-output example pairs" is constructed for each target chapter. At the same time, through inductive analysis of these "input information" (automatic statistics and manual review), a set of "common localized input features" (e.g. in writing "differential diagnosis", a certain specialty hospital emphasizes the exclusion of some rare diseases in addition to general logic, which constitutes part of the localized features) that are usually needed when generating the content of the specific target chapter for the institution are identified. The output of this step is a structured example pair set rich in the practice experience of the institution and a key input feature list for each target chapter.

[0066] Step c: Based on the characteristics of the target chapter, input-output example pairs, and localized input features, combined with pre-set prompt engineering strategies (such as role instruction design, output format constraints, etc.), semi-automatically generate an initial configuration of the intelligent agent, including prompt templates with dynamic data placeholders and example areas;

[0067] Prompt engineering strategies include but are not limited to at least one of the following: role-based instruction set design, task decomposition-based guided instruction sequence design, output style and format constraint instruction design for specific medical document types, and arrangement strategies for the number, order, and diversity of examples in in-context learning;

[0068] Using prompt engineering auxiliary tools, the generated prompt templates are parameterized, managed, compared and tested in multiple versions, tracked for effectiveness based on evaluation indicators, and structured and debugged for complex prompt logic;

[0069] The prompt word template can accommodate 1 to 5 pairs of structured input-output examples, and the selection and combination of example pairs can be dynamically adjusted according to the context window limit;

[0070] For each target chapter, an initial "configuration" of a large language model agent is semi-automatically designed and generated, the core of this "configuration" is a highly customized prompt word template, which contains placeholders for dynamically filling specific data of new cases (the naming and structure of which refer to the aforementioned identified "common localized input features"), more importantly, it contains preset areas and logic for dynamically and strategically embedding one or more high-quality "input-output example pairs" selected from the local case library for effective context learning, the instruction part of the prompt word (such as role setting, task description, output requirement, etc.) is also preliminarily set according to the characteristics of the target chapter and the needs of the institution, for example, the system may automatically recommend a prompt word structure containing 2 or 3 examples according to the complexity of the target chapter, all designed prompt word templates (i.e. the core of the agent configuration) are included in version management and continuously iteratively optimized through subsequent actual use and feedback.

[0071] Step d: When processing a new case, select the initial configuration of the agent corresponding to the target chapter, from the constructed input-output example pairs, use the preset filtering strategy (such as department matching, disease similarity, etc.) to select and create a set of example pairs that match the new case from the example pairs, fill the new case input data into the placeholders of the prompt word template, fill the selected example pair set into the example area, form the final execution prompt word, call the large language model, interact using the final execution prompt word, and obtain the draft content of the target chapter;

[0072] After the doctor (or the system through EMR integration) selects the document type and target chapter to be processed, the "large language model agent configuration" (i.e. the corresponding prompt word template and related strategies) previously customized for this chapter of the institution will be loaded from the library, then the most suitable examples will be selected from the structured input-output example pair library of the institution according to the preset filtering strategy;

[0073] Then, the real-time clinical data of the new case is accurately filled into the corresponding placeholders of the prompt word template, and the selected context learning examples are also embedded into the designated positions of the template, in this way, a complete and highly targeted "final execution prompt word" for the institution, the chapter, and the new case is dynamically constructed, this prompt word is then sent to the underlying large language model service, after understanding this prompt word containing rich local context and explicit instructions, the LLM generates the draft content for the target chapter of the new case.

[0074] Specifically:

[0075] Extracting clinical data from the input information of the constructed input-output example pairs, wherein the clinical data includes diagnosis information, symptom information, patient age, and gender;

[0076] Setting the age interval corresponding to each age group, and converting the patient age of the new case and the patient age of the example pair into the corresponding age interval;

[0077] Mapping the diagnosis information of the new case patient and the example pair patient to ICD-10 code (such as "type 2 diabetes" -> E11, "hypertension" -> I10);

[0078] If the gender, age interval, and ICD-10 code of a certain example pair patient are consistent with the gender, age interval, and ICD-10 code of the new case patient, it is determined as a successful match, and the example pair that matches the new case successfully is retained as a candidate example pair;

[0079] Extracting symptom performance segments from the symptom information of the new case patient and the candidate example pair patient, and performing standardization processing using a medical terminology dictionary (such as "sore throat" -> "sore throat"); After processing, the symptom word set of the new case patient and the symptom word set of the candidate example pair patient are constructed;

[0080] For the symptom word set of the new case patient and the symptom word set of the candidate example pair patient, perform symptom word matching, count the number of matching symptom words as the matching number, calculate the proportion of the matching number in the total number of new case symptom words, and obtain the symptom matching ratio;

[0081] It is supplemented that by strict matching of gender, age interval, and ICD-10 code, the example pair that is highly consistent with the new case in terms of demographic characteristics (gender, age) and core diagnosis is preferentially retained. For example, for a new case of "65-year-old male acute myocardial infarction (I21.0)", the example pair of "50-year-old female diabetes (E11)" can be quickly excluded to avoid irrelevant cases and ensure that the basic clinical characteristics of the candidate example pair match the new case.

[0082] Symptom standardization and quantitative matching reduce ambiguity: Through the medical terminology dictionary, the symptoms are standardized (such as "sore throat" -> "sore throat"), eliminating the ambiguity of natural language expressions; and through the "symptom matching rate", the degree of symptom overlap is quantified, which can accurately screen out example pairs with highly similar symptoms to the new case.

[0083] Extracting the utility data of the candidate example pair; the utility data includes historical call frequency, historical feedback score, and modification character proportion;

[0084] The historical calling times of the candidate example pair are counted, and each group of time intervals corresponding to the historical calling times is constructed in advance, and each group of time intervals corresponds to a content performance score; wherein the content performance score is set in the range of 1-10 and is a positive integer; the higher the historical calling times, the higher the corresponding content approval score obtained by matching; the historical calling times of the candidate example pair are matched with the corresponding time intervals to determine the content performance score of the candidate example pair; the historical feedback score of each calling of the candidate example pair is obtained, and the historical feedback score is set in the range of 1-10 and is a positive integer; the historical feedback score is scored by the satisfaction of the doctor to the generated content, and the higher the historical feedback score, the higher the satisfaction of the doctor to the generated content; the average value of each group of historical feedback scores of the candidate example pair is taken to obtain the content satisfaction score of the candidate example pair; the content performance score and the content satisfaction score of the candidate example pair are summed to obtain a content comprehensive score, and the highest content comprehensive score is extracted from the constructed input-output example pair; the content comprehensive score of the candidate example pair is divided by the highest content comprehensive score to obtain a content approval ratio;

[0085] It is further explained that the relationship between “frequency of use” and “doctor satisfaction” is balanced by the summation calculation of “content performance score + content satisfaction score”, for example, a certain example pair may have high calling times due to wide applicable scenarios (high content performance score), but the doctor has low satisfaction with the generated content (low content satisfaction score), and the comprehensive score can avoid the misselection caused by relying on a single indicator;

[0086] The content approval ratio enables comparison across example pairs: by calculating “candidate example pair comprehensive score ÷ highest comprehensive score”, the score is standardized to a ratio value of 0-1, which directly reflects the relative quality of the candidate example pair in all examples, for example, an example pair with a content approval ratio of 0.9 is more likely to generate high-quality content than an example pair with a content approval ratio of 0.6, which facilitates priority selection.

[0087] The modification character ratio of the generated content after each calling of the candidate example pair is extracted (the ratio of characters modified by the doctor), and the average value is taken as the content modification ratio of the candidate example pair;

[0088] The symptom matching ratio, the content approval ratio, and the content modification ratio of the candidate example pair are respectively marked as ; according to the formula The symptom matching ratio, the content approval ratio, and the content modification ratio of the candidate example pair are respectively marked as ; wherein The symptom matching ratio, the content approval ratio, and the content modification ratio are respectively marked as

[0089] The screening evaluation index of each candidate example pair is , are compared with the preset screening reference index, if there are multiple candidate example pairs of screening evaluation indexes higher than the screening reference index, the candidate example pairs are sorted from large to small according to the size of the screening evaluation index , and the first three candidate example pairs are selected from left to right to construct the example pair set;

[0090] If the screening evaluation index of each candidate example pair is lower than the preset screening reference index, the candidate example pair with the higher screening evaluation index is directly selected to construct the example pair set;

[0091] It is to be noted that the example pair set is constructed by the calculated screening evaluation index, so that the selected example pairs have the characteristics of "similar clinical features", "good historical performance" and "less modification", which can provide more accurate reference for the large language model, so that the generated medical record content is more in line with the actual situation of the new case and more in line with the doctor's writing habits, thereby reducing the doctor's modification cost.

[0092] Step e: presenting the draft content to the user for review, editing and confirmation, and collecting user feedback information on the draft content; the feedback information includes satisfaction, etc.

[0093] Embodiment 2

[0094] Please refer to Figure 2 Based on the personalized medical record-oriented large language model agent construction method provided in Embodiment 1 of the present application, Embodiment 2 of the present application proposes a personalized medical record-oriented large language model agent construction system. Embodiment 2 is only a preferred way of Embodiment 1, and the implementation of Embodiment 2 will not affect the implementation of Embodiment 1 alone.

[0095] Specifically, the personalized medical record-oriented large language model agent construction system provided in Embodiment 2 of the present application is different in that it includes:

[0096] Target identification module: obtain the preset medical record template of the target medical institution, extract the chapter level, title and fixed text of the template, and determine the target chapter generated by the large language model agent based on the content characteristic data analysis result of the chapter;

[0097] ​Case processing module: obtain high-quality anonymized medical record cases corresponding to the target chapter from the target medical institution pre-labeled, the case contains the input information of the training agent and the reference output content of the target chapter, extract the input information and reference output content from the medical record case, construct a structured input-output example pair, analyze the input information, identify the target department corresponding to the target chapter, extract the pre-integrated diagnosis and treatment terminology set of the target department as the localized input features required for generating the target chapter;

[0098] Initial setting module: based on the target chapter features, input-output example pairs and localized input features, combined with the preset prompt word engineering strategy, semi-automatically generate the initial configuration of the agent, including the prompt word template of the dynamic data placeholder and the example area;

[0099] Case processing module: when processing a new case, select the initial configuration of the agent corresponding to the target chapter, from the constructed input-output example pairs, use the preset screening strategy to select and create a set of example pairs matching the new case from the example pairs, fill the new case input data into the placeholder of the prompt word template, fill the selected example pair set into the example area, form the final execution prompt word, call the large language model, interact using the final execution prompt word, and obtain the draft content of the target chapter;

[0100] Result confirmation module: present the draft content to the user for review, editing and confirmation, and collect user feedback information;

[0101] The above formulas are all dimensionless values calculated, and specific dimensionless methods such as standardization can be used, which will not be described here. The formula is obtained by software simulation of a large amount of data to obtain a formula of the nearest real situation, and the preset parameters in the formula are set by the person skilled in the art according to the actual situation.

[0102] The above-described embodiments can be implemented in part or in whole through software, hardware, firmware or any combination thereof. When implemented in software, the above-described embodiments can be implemented using one or more computer programs written in any suitable programming language. Such programs can be stored in one or more storage media or memory devices (e.g., a computer readable medium) associated with the computer or other suitable devices. The memory devices can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other suitable memory devices. The computer programs can be loaded and / or executed on the computer or other suitable devices to produce a computer implemented process, such that the actions specified in the computer programs are performed. The computer programs can be executed on a single computer or on multiple computers.

[0103] It should be understood that the sequence of the above processes is not intended to mean the execution order of the processes, and the execution order of the processes should be determined according to the functions and inherent logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0104] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0106] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, which may be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0107] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit.

[0108] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile ATA hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0109] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for constructing a large language model intelligent agent for personalized medical documents, characterized in that, include: Local processing: Based on the target chapter generated by the large language model, high-quality anonymized medical records from the target medical institution are obtained, which correspond to the target chapter. The records contain the input information of the training agent and the reference output content of the target chapter. The input information and reference output content are extracted from the medical records to construct structured input-output example pairs. The input information is analyzed to identify the target department corresponding to the target chapter and to extract the pre-integrated set of diagnostic and treatment terms of the target department as the localized input features required to generate the target chapter. Initial setup: Based on target chapter features, input-output example pairs, and localized input features, combined with preset prompt word engineering strategies, the initial configuration of the agent is generated semi-automatically, including dynamic data placeholders and prompt word templates for the example area; Case processing: When processing a new case, select the initial configuration of the agent corresponding to the target chapter, select and create a set of example pairs that match the new case from the pre-built input-output example pairs using a preset filtering strategy, fill the placeholders of the prompt word template with the input data of the new case, fill the example area with the selected set of example pairs to form the final execution prompt word, call the large language model, use the final execution prompt word to interact, and obtain the draft content of the target chapter.

2. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 1, characterized in that, The process of selecting and creating a set of example pairs that match the new case from the example pairs using a preset filtering strategy specifically involves: Extract clinical data from the input information of the constructed input-output example pairs, where the clinical data includes diagnostic information, symptom information, patient age, and gender; and filter candidate example pairs from the constructed input-output example pairs based on the clinical data. The clinical and utility data of the candidate example pairs are analyzed and processed to construct a set of example pairs that match the new cases; the utility data includes the number of historical calls, historical feedback scores, and the proportion of modified characters.

3. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 2, characterized in that, The process of selecting candidate example pairs from the constructed input-output example pairs based on clinical data is as follows: Define the age range for each age group, and convert the ages of new case patients and example patient ages into the corresponding age ranges; For new cases and example patients, the diagnostic information will be mapped to ICD-10 codes; If the gender, age range, and ICD-10 code of a sample patient match the gender, age range, and ICD-10 code of a new case patient, then the match is considered successful, and the sample pair that matches the new case is retained as a candidate sample pair.

4. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 3, characterized in that, The analysis and processing of clinical data for the candidate example pairs specifically includes: Symptom segments are extracted from the symptom information of new case patients and candidate example patients, and standardized using a medical terminology dictionary; after processing, a set of symptom terms for new case patients and a set of symptom terms for candidate example patients are constructed. For the set of symptom words of new patients and the set of symptom words of candidate examples, symptom word matching is performed. The number of symptom words that match is counted as the number of matches. The proportion of the number of matches to the total number of symptom words in new patients is calculated to obtain the symptom matching ratio.

5. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 4, characterized in that, The analysis and processing of the utility data for candidate example pairs specifically includes: The historical call counts of candidate example pairs are counted. Intervals corresponding to each historical call count are pre-constructed, with each interval corresponding to a content performance score. The historical call counts of candidate example pairs are matched with their corresponding intervals to determine the content performance score. Historical feedback scores after each call to the candidate example pair are obtained. The average of the historical feedback scores for each group of candidate example pairs is taken to obtain the content satisfaction score. The content performance score and content satisfaction score of the candidate example pair are summed to obtain the comprehensive content score. The highest comprehensive content score is extracted from the constructed input-output example pairs, and the comprehensive content score of the candidate example pair is divided by the highest comprehensive content score to obtain the content acceptance ratio. The percentage of modified characters in the generated content after each historical call of the candidate example pair is extracted and averaged as the content modification ratio of the candidate example pair.

6. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 5, characterized in that, The construction of the set of example pairs matched with new cases is specifically as follows: The symptom matching ratio, content acceptance ratio, and content modification ratio of candidate example pairs are comprehensively processed to determine the screening evaluation index of candidate example pairs. The screening evaluation index of each candidate example pair is compared with the preset screening reference index. If there are multiple candidate example pairs with screening evaluation indices higher than the screening reference index, the candidate example pairs are sorted from largest to smallest according to the size of the screening evaluation index, and the top three candidate example pairs are selected from left to right to construct the example pair set. If the screening evaluation index of each candidate example pair is lower than the preset screening reference index, then the candidate example pair with the higher screening evaluation index is directly selected to construct the example pair set.

7. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 1, characterized in that, The method for constructing a large language model agent also includes: Target identification: Obtain the preset document template of the target medical institution, extract the chapter level, title and fixed text of the template, and determine the target chapter generated by the large language model agent based on the content characteristic data analysis results of the chapter; the content characteristic data includes the proportion of descriptive text, writing time and cost and the degree of structuring.

8. The method for constructing a large language model intelligent agent for personalized medical documents according to claim 7, characterized in that, The determination of the target chapter generated with the assistance of the large language model agent specifically includes: Extract the proportion of descriptive text in the chapter content; combine the doctor's work records to calculate the average writing time of each chapter; compare the proportion of descriptive text and the average writing time of each chapter with the preset proportion reference value and writing reference time, and select chapters with both the proportion of descriptive text and the average writing time higher than the proportion reference value and the writing reference time as candidate chapters; Retrieve the standardized entity description fields contained in the candidate chapters, calculate the proportion of the number of standardized entity description fields in the candidate chapters to the total number of fields in the chapters, and obtain the standardization rate of the candidate chapters. The standardization rate of candidate chapters is matched with the pre-defined range of standardization rates corresponding to the medium structure standardization rate. If a match is successful, the chapter is used as the target chapter generated by the large language model agent.

9. A large language model intelligent agent construction system for personalized medical documents, applied to the large language model intelligent agent construction method for personalized medical documents as described in any one of claims 1-8, characterized in that, include: Target identification module: Obtains the preset document template of the target medical institution, extracts the chapter level, title and fixed text of the template, and determines the target chapter generated by the large language model agent based on the data analysis results of the content characteristics of the chapter; Case processing module: Acquires high-quality anonymized medical record cases from the target medical institution that correspond to the target chapter. The cases contain the input information for training the agent and the reference output content of the target chapter. Extracts the input information and reference output content from the medical record cases, constructs structured input-output example pairs, analyzes the input information, identifies the target department corresponding to the target chapter, and extracts the pre-integrated set of diagnostic and treatment terms for the target department as the localized input features required to generate the target chapter. Initial setup module: Based on target chapter features, input-output example pairs and localized input features, combined with preset prompt word engineering strategies, it semi-automatically generates the initial configuration of the agent, including dynamic data placeholders and prompt word templates for the example area; Case processing module: When processing a new case, select the initial configuration of the agent corresponding to the target chapter, select and create a set of example pairs that match the new case from the pre-built input-output example pairs using a preset filtering strategy, fill the placeholders of the prompt word template with the input data of the new case, fill the example area with the selected set of example pairs to form the final execution prompt word, call the large language model, use the final execution prompt word to interact, and obtain the draft content of the target chapter; Result Confirmation Module: Presents draft content to users for review, editing, and confirmation.

Images