Medical care intelligent form generation method and system based on multi-modal large model

Abstract

The present application relates to the medical care form technical field, propose a kind of medical care intelligent form generation method and system based on multimodal big model, comprising: collecting the physical examination of clinical business system in medical care patient, diagnosis record, diagnosis voice data and other multimodal data;Again, generate physical change sequence by multimodal big model in combination with physical examination data, construct diagnosis feature vector;Subsequently, the vector is mapped to medical quality index system, to generate visual form framework;Then, extract diagnosis record key elements and fill into framework, update form information in combination with diagnosis voice data;Finally, medical care intelligent form adapted to each system is output by multimodal big model, and is synchronized to medical care work terminal.The present application realizes the effective fusion of multimodal medical data, and the generated form can completely reflect the whole picture of diagnosis and treatment and the dynamic evolution of signs, improve the clinical work efficiency and medical quality.

Description

Technical Field

[0001] This invention relates to a method and system for generating intelligent medical and nursing forms based on a multimodal large model, belonging to the field of medical and nursing form technology. Background Technology

[0002] Intelligent forms for medical care are an intelligent approach that uses multimodal big data technology to comprehensively process multi-dimensional data during the diagnosis and treatment process, and automatically build and output structured medical documents. It is of great significance for improving clinical work efficiency and ensuring medical quality.

[0003] Currently, hospitals generally rely on separate clinical business systems to record patient information. Form generation is mostly done manually by summarizing and filling in forms, or partially automated by using rule templates based on a single data type. These existing methods have significant limitations: medical data from different sources and with different modalities are difficult to effectively integrate, analyze, and correlate, resulting in fragmented form content that cannot fully and coherently reflect the overall picture of diagnosis and treatment and the dynamic evolution of the patient's vital signs. Therefore, the above problems restrict the automation of intelligent medical form generation and its business adaptability. Summary of the Invention

[0004] This invention provides a method and system for generating intelligent medical forms based on a multimodal large model. Its main purpose is to achieve effective integration of multimodal medical data. The generated forms can fully reflect the overall picture of diagnosis and treatment and the dynamic evolution of physical signs, thereby improving clinical work efficiency and medical quality.

[0005] To achieve the above objectives, this invention provides a method for generating intelligent medical forms based on a multimodal large model, comprising: Acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data; Using a pre-defined multimodal large model combined with the vital sign test data, a sequence of vital sign changes of the medical staff and patients during the consultation process is generated, and a diagnostic feature vector with the sequence of vital sign changes as the core is constructed. After mapping the diagnostic feature vectors to the medical quality indicator system of each clinical business system, a mapping indicator set is obtained, and a visual form framework under each clinical business system is generated based on the key indicator items in the mapping indicator set. Extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form; Based on the diagnostic voice data, update the medical and nursing information entries in the fully filled form, and combine them with the preset multimodal large model to output a medical and nursing intelligent form that conforms to each clinical business system, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0006] Optionally, the step of using a preset multimodal large model combined with the vital sign test data to generate a sequence of vital sign changes in the medical staff and patients during the consultation process includes: Generate an initial set of vital signs in the continuous detection process of the vital sign test data, and extract time-series vital sign pairs containing the vital sign collection time and specific vital sign values ​​from the initial set of vital signs; After analyzing the trend segments of the vital signs corresponding to the values ​​of the vital signs in the time series, the time intervals associated with the trend segments of the vital signs are connected to obtain the segments of changes in vital signs. The numerical deviation records in the segments of the changes in vital signs are corrected, and based on the corrected segments, a sequence of changes in vital signs of the medical staff and patients during the consultation process is constructed.

[0007] Optionally, the segment of vital sign change represents a continuous time data unit formed by identifying the changing trend of a specific physiological indicator from continuously detected vital sign data. It consists of several temporally adjacent and trend-consistent vital sign measurements and their corresponding time points, representing the relatively complete physiological state evolution process of the patient within a certain period of time.

[0008] Optionally, acquiring multimodal data of medical staff and patients in various clinical business systems includes: After parsing the patient identifiers of the medical staff and patients in each clinical business system, a secure data link is established corresponding to the access node under each clinical business system. Collect multi-mode data streams from the secure data link and perform format alignment on the multi-mode data streams; The aligned data stream is aggregated according to the timestamp of the patient identifier to obtain multimodal data.

[0009] Optionally, after mapping the diagnostic feature vectors to the medical quality indicator systems in each clinical business system, a mapping indicator set is obtained, including: Analyze the clinical feature itemset in the diagnostic feature vector; Based on the clinical feature item set, the medical quality indicator system in each clinical business system is invoked to obtain the indicator item list of the business indicator library under the medical quality indicator system. After comparing the set of clinical features with the list of indicator items item by item, a group of associated indicator items is obtained, and the associated indicator items group is mapped into the mapping indicator set in the medical quality indicator system.

[0010] Optionally, generating the visual form framework for each clinical business system based on the key indicator items in the mapping indicator set includes: The clinical records and indicator ranges corresponding to the key indicator items in the mapping indicator set are analyzed. Based on the clinical records and indicator ranges, configure the chart types and data display areas in the preset forms to form a preliminary framework outline; Based on the preliminary framework outline and the form specifications of each clinical business system, a visual form framework for each clinical business system is generated.

[0011] Optionally, filling the diagnostic elements into the visual form frame according to a preset form item template to obtain a fully filled form includes: Retrieve the sequence of item names and corresponding content format requirements from the preset form item template; Extract the specific content from the key diagnostic and treatment elements that matches the sequence of entry names; After standardizing the specific content according to the content format requirements, the standardized content, combined with the semantic information, is loaded into the corresponding item position in the visual form frame. After confirming that the content embedded in the corresponding entry position carries the key diagnostic and treatment elements, a fully populated form is generated.

[0012] Optionally, updating the medical information entries in the fully populated form based on the diagnostic voice data includes: Identify the speech feature descriptions in the diagnostic speech data and convert the speech feature descriptions into an audit guide list associated with the form entries in the fully filled form; Verify the record information of each item in the audit guidelines list; When there is a discrepancy between the entry record information and the speech feature description, the entry record information is marked as being in a state to be corrected. Update the healthcare information entries in the fully populated form under the state to be corrected.

[0013] Optionally, the review guide list is a standardized checklist generated by converting speech feature descriptions in diagnostic speech data and associating them with specific entries in a fully filled form. This list maps image language descriptions in images to structured review items for form entries, providing a clear target list for item-by-item verification.

[0014] To address the aforementioned problems, this invention also provides a medical and nursing intelligent form generation system based on a multimodal large model, the system comprising: The multimodal data module is used to acquire multimodal data of medical staff and patients in various clinical business systems. The multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data. The feature vector module is used to generate a sequence of changes in the vital signs of the medical staff and patients during the consultation process by combining the preset multimodal large model with the vital sign test data, and to construct a diagnostic feature vector with the sequence of changes in vital signs as the core. The form framework module is used to map the diagnosis and treatment feature vectors to the medical quality indicator system of each clinical business system to obtain the mapping indicator set, and generate a visual form framework under each clinical business system based on the key indicator items in the mapping indicator set. The form filling module is used to extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form. The intelligent form module is used to update the medical and nursing information entries in the fully filled form based on the diagnostic voice data, and output the medical and nursing intelligent form that conforms to the clinical business systems by combining the preset multimodal large model, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0015] Compared to the problems described in the background art, the embodiments of the present invention can integrate patient diagnosis and treatment information scattered across different business systems. Furthermore, the multimodal data encompassing vital sign examination data, in-hospital medical records, and diagnostic voice data can comprehensively present various key states during the diagnosis and treatment process. Next, the embodiments of the present invention can integrate the scattered vital sign examination data, analyze the dynamic changes in vital sign indicators during the patient's visit, and clearly present the evolution of each vital sign indicator during the patient's visit, providing structured data support for the generation of intelligent medical forms. Then, after mapping the diagnosis and treatment feature vectors to the medical quality indicator systems of each clinical business system, the embodiments of the present invention can connect the diagnosis and treatment feature vectors with… Clinical quality standards are established to form a set of mapping indicators with clear clinical orientation, thereby clarifying the core content of forms under various clinical business systems. Furthermore, embodiments of this invention can integrate core diagnostic and treatment information scattered in medical records, ensuring precise matching between form content and framework structure. The key diagnostic and treatment elements filled in enrich the core content of the framework, clarifying the diagnostic and treatment basis corresponding to each item in the form. Furthermore, embodiments of this invention can supplement image-related diagnostic and treatment information in the form, improving the core content of the form. The intelligent medical and nursing form is synchronized to the medical and nursing work terminal, enabling the key link in data transmission to be opened up, allowing medical staff to directly obtain the integrated diagnostic and treatment information, providing comprehensive information support for clinical work. Therefore, this invention achieves effective fusion of multimodal medical data, and the generated form can completely reflect the overall picture of diagnosis and treatment and the dynamic evolution of vital signs, improving clinical work efficiency and medical quality. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for generating intelligent medical forms based on a multimodal large model, according to an embodiment of the present invention. Figure 2 A schematic diagram of the modules for implementing a medical and nursing intelligent form generation system based on a multimodal large model, provided in an embodiment of the present invention; Figure 3 A schematic diagram of a computer device for a method of generating intelligent medical forms based on a multimodal large model, according to an embodiment of the present invention; The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] This application provides a method for generating intelligent medical and nursing forms based on a multimodal large model. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, this method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0019] Reference Figure 1 The diagram shown is a flowchart illustrating a method for generating intelligent medical forms based on a multimodal large model, according to an embodiment of the present invention. In this embodiment, the method for generating intelligent medical forms based on a multimodal large model includes: S1. Acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data.

[0020] The embodiments of the present invention can integrate patient diagnosis and treatment information scattered in different business systems, and the multimodal data covering vital sign test data, in-hospital diagnosis and treatment records and diagnostic voice data can fully present various key states during the diagnosis and treatment.

[0021] The clinical business systems refer to a collection of specialized information systems within a medical institution that support the operation of different clinical processes, encompassing multiple subsystems such as electronic medical record management systems, laboratory result reporting systems, medical image storage systems, and outpatient registration and payment systems. The medical staff and patients refer to individuals who have established formal medical service relationships with the medical institution, including outpatients, inpatients, and those undergoing physical examinations. These individuals generate various information related to their health status, such as vital signs, laboratory tests, and imaging data, during diagnosis, treatment, or health checkups. The multimodal data refers to the collection of different types and forms of medical-related information collected by the various clinical business systems during the diagnosis and treatment process. Specifically, it includes structured vital sign and laboratory test data in numerical and tabular formats, semi-structured in-hospital medical records in text and document formats, and unstructured diagnostic voice data in voice format.

[0022] As an embodiment of the present invention, the acquisition of multimodal data of medical staff and patients in various clinical business systems includes: parsing the patient identifiers of the medical staff and patients in each clinical business system, establishing a secure data link corresponding to the access node in each clinical business system; collecting multimodal data streams in the secure data link and performing format alignment on the multimodal data streams; and aggregating the aligned data streams according to the identification timestamp of the patient identifier to obtain multimodal data.

[0023] The patient identifier refers to a set of character codes uniformly assigned by the medical institution to uniquely identify the identity of medical staff and patients. This code contains basic patient identity information and information related to medical records, and can be recognized and parsed by various clinical business systems. It is the core identification basis for matching data of the same patient at different stages of treatment across systems. The access node refers to the interface endpoint set up inside each clinical business system to provide data access services to the outside world. Each access node corresponds to a data output module of a clinical business system and has data read permission verification function. It is the docking carrier for establishing data interaction connection between external devices and clinical business systems. The secure data link refers to the communication channel built based on the data encryption transmission protocol to connect external data acquisition devices and access nodes of each clinical business system. This link has identity verification... The system includes data tamper-proofing features, which encrypt data in real time during transmission to prevent data leakage or illegal tampering. The multi-mode data stream refers to the raw, unformatted data sequence collected in real time from access nodes of various clinical business systems. This data stream comprises three types of parallel-transmitted sub-data streams: structured vital sign numerical data stream, semi-structured in-hospital medical record data stream, and unstructured diagnostic voice data stream. Each sub-data stream exists independently and is not integrated. The identifier timestamp refers to a time information tag generated in each clinical business system along with the multi-mode data stream, used to record the time of data generation. This timestamp is uniquely bound to the patient identifier and uses a standardized time encoding format to accurately record the time of data generation or collection in the clinical business system.

[0024] Specifically, the multimodal data includes vital sign examination data, in-hospital medical records, and diagnostic voice data. The vital sign examination data refers to structured data collected and generated by clinical laboratory equipment or monitoring instruments, primarily presented in numerical or tabular form, covering the patient's complete blood count indicators, biochemical test results, vital sign parameters, pathogen detection data, etc. The in-hospital medical records refer to a collection of texts recorded by medical staff during the patient's visit to a medical institution, documenting the entire treatment process according to clinical treatment guidelines, including outpatient medical records, inpatient progress notes, doctor's orders, and examination requests. Please refer to reports, surgical records, consultation opinions, etc. These records are divided into structured and semi-structured types, linked to the patient's unique identifier, and stored in the diagnosis and treatment database of each clinical business system. The diagnostic voice data refers to unstructured data, mainly in audio streams, recorded by medical staff at key stages of clinical diagnosis and treatment. It covers oral descriptions of the patient's condition during ward rounds, statements on image diagnosis and analysis, voice records of case discussions, and recordings of verbal instructions from doctors. This data is stamped with timestamps, speaker identifiers, and tags associated with the patient's diagnosis and treatment process, and is stored and managed using a medical voice acquisition system.

[0025] S2. Using a preset multimodal large model combined with the vital sign test data, generate a sequence of vital sign changes of the medical staff and patients during the consultation process, and construct a diagnostic feature vector with the sequence of vital sign changes as the core.

[0026] The embodiments of the present invention can integrate scattered vital sign test data, sort out the dynamic change trajectory of vital sign indicators during the medical treatment process, and clearly present the evolution of each vital sign indicator during the patient's medical treatment, providing structured data support for the generation of intelligent medical forms.

[0027] The pre-set multimodal large model refers to a deep learning model pre-trained and fine-tuned based on a multimodal labeled dataset in the medical field and deployed in the medical and nursing intelligent form generation system. This model has the ability to extract features, perform time series analysis, and model correlations from structured vital sign test data. It can directly receive standardized data input from the system and output corresponding analysis results. The model parameters remain fixed after deployment. The vital sign change sequence refers to a set of vital sign indicator changes arranged in chronological order, generated by the pre-set multimodal large model after analyzing the vital sign test data. This sequence is bound to each time point of the patient's visit and includes the values, fluctuation ranges, and trends of various vital sign indicators at different stages of diagnosis and treatment. Each record in the sequence is associated with the corresponding source of vital sign test data. The diagnosis and treatment feature vector refers to a multidimensional vector structure formed by quantifying features through the pre-set multimodal large model with the vital sign change sequence as the core. Each dimension of the vector corresponds to the change feature of a core vital sign indicator, and the dimension value is generated by mapping parameters such as the change amplitude and duration of the vital sign indicator.

[0028] As an embodiment of the present invention, the step of using a preset multimodal large model combined with the vital sign test data to generate a sequence of vital sign changes for the medical staff and patients during the consultation process includes: generating an initial set of vital signs in the continuous detection process of the vital sign test data, and extracting time-series vital sign pairs containing the vital sign collection time and specific vital sign values ​​from the initial set of vital signs; analyzing the vital sign trend segments corresponding to the vital sign values ​​in the time-series vital sign pairs, and then connecting the time intervals associated with the vital sign trend segments to obtain vital sign change segments; correcting the numerical deviation records in the vital sign change segments, and constructing a sequence of vital sign changes for the medical staff and patients during the consultation process based on the corrected segments.

[0029] The initial vital sign set refers to the dataset formed after preliminary integration of all vital sign test data generated during continuous testing. This dataset contains all raw data obtained from one or more vital sign tests. The vital sign acquisition time refers to the specific point in time when the clinical testing equipment or monitoring instrument completes the testing of one or more vital sign indicators of the patient. This time point is recorded using standardized time coding rules and bound to the patient identifier and the corresponding vital sign test item, accurately marking the execution time of a single vital sign test. The specific vital sign value refers to the quantitative result output by the clinical testing equipment or monitoring instrument after testing a specific vital sign indicator of the patient. This value corresponds to a clearly defined... The types of vital signs, such as body temperature, heart rate, and blood glucose concentration, are measured in units that follow general clinical laboratory standards and correspond one-to-one with the time of data collection. The temporal vital sign pairs refer to binary data structures extracted from the initial set of vital signs, consisting of a set of interrelated data collection times and specific value values. Each temporal vital sign pair corresponds to the time and value information of a single vital sign test, arranged in chronological order of data collection time, and serves as the basic data unit for analyzing trend segments of vital signs. The vital sign values ​​refer to the quantitative data in the temporal vital sign pair that corresponds to the time of data collection and characterizes the patient's vital sign status. This data directly originates from the original test results of the vital sign examination data. Without trend analysis or bias correction, the numerical values ​​reflect the actual measured status of the patient's vital signs at the corresponding time point. The "vital sign trend segment" refers to a data segment obtained after continuous analysis of the vital sign values ​​in a time-series vital sign pair. This data segment corresponds to a continuous time range and can characterize the changing trend of a certain vital sign indicator within that time period, such as an upward trend, a downward trend, or a stable trend. The "time interval" refers to the continuous time range corresponding to the vital sign trend segment, with the start time being the time of data collection for the first time-series vital sign pair included in the trend segment and the end time being the time of data collection for the last time-series vital sign pair. The span of the time interval is related to the duration of the vital sign trend segment. The duration is consistent; the vital sign change segment refers to a continuous data segment formed by connecting related vital sign trend segments and their corresponding time intervals. This segment contains all time-series vital sign pairs and vital sign trend segment information within the corresponding time interval. It has not undergone numerical deviation correction processing and can completely reflect the change process of vital sign indicators within a certain time period. The numerical deviation record refers to an abnormal data record in the initial vital sign set, time-series vital sign pairs, or vital sign trend segments that does not conform to the conventional detection value range or change pattern of the vital sign indicator. The causes of this record include detection equipment error, acquisition operation deviation, data transmission error, etc., and it is an abnormal data item that needs to be corrected.

[0030] Furthermore, as another embodiment of the present invention, the vital sign change segment represents a continuous time data unit formed by identifying the changing trend of a specific physiological indicator from continuously detected vital sign data. It is composed of several temporally adjacent and trend-consistent vital sign measurement values ​​and their corresponding time points, characterizing the relatively complete physiological state evolution process of the patient within a time period.

[0031] In detail, the construction of the diagnostic feature vector with the vital sign change sequence as the core can be achieved by sorting the acquired discrete "vital sign test data", such as multiple records of body temperature, blood pressure, heart rate, blood oxygen saturation, laboratory index values, etc., by timestamp to form a structured "vital sign change sequence". For each single point data in the sequence, a preset multimodal large model is used, such as a Visual-Language Model that has been fine-tuned with medical corpus and numerical data, such as the text / numerical understanding branch of a variant of Florence-2 or Med-PaL MM, to convert it into a static feature vector.

[0032] S3. After mapping the diagnosis and treatment feature vectors to the medical quality indicator system of each clinical business system, a mapping indicator set is obtained, and a visual form framework under each clinical business system is generated based on the key indicator items in the mapping indicator set.

[0033] After mapping the diagnostic and treatment feature vectors to the medical quality indicator systems of each clinical business system, the embodiments of the present invention can connect the diagnostic and treatment feature vectors with clinical quality standards to form a mapping indicator set with clear clinical orientation, thereby clarifying the core content of the forms under each clinical business system.

[0034] The medical quality indicator system refers to a set of indicators developed by medical institutions in accordance with clinical treatment guidelines and business management requirements, applicable to various clinical business systems. This system covers multiple dimensions of indicators, including treatment compliance, accuracy of test results, and completeness of vital sign monitoring. Each indicator has a clear definition, value range, and coding rules. The mapping indicator set refers to the set of indicators obtained by matching and associating the features of each dimension of the treatment feature vector with the indicators in the medical quality indicator system. This set contains all indicators that successfully match the treatment feature vector, and each indicator has a label corresponding to the dimension of the feature vector. The key indicators are those selected from the mapping indicator set that are directly related to the core treatment needs of each clinical business system. The selection criteria are the importance of the indicators in clinical treatment decisions and patient status assessments. These indicators cover the key aspects of the entire patient visit process and are the core data elements for constructing the visual form framework. The visual form framework is a form structure template with visual presentation capabilities built based on the key indicators. This framework includes indicator classification hierarchy, item arrangement order, data display format, and form layout style.

[0035] As an embodiment of the present invention, the step of mapping the diagnostic feature vector to the medical quality indicator system of each clinical business system to obtain the mapped indicator set includes: parsing the clinical feature item set in the diagnostic feature vector; based on the clinical feature item set, calling the medical quality indicator system in each clinical business system to obtain the indicator item list of the business indicator library under the medical quality indicator system; comparing the clinical feature item set with the indicator item list item by item to obtain the associated indicator item group, and mapping the associated indicator item group to the mapped indicator set in the medical quality indicator system.

[0036] The clinical feature itemset refers to the feature set obtained by decomposing and classifying the features of each dimension of the diagnostic and treatment feature vector. Each feature item in this set corresponds to a specific feature related to patient diagnosis and treatment, covering types such as features of changes in vital signs and features of the treatment stage. Each feature item has a clear feature definition and dimension identifier, and is the basic unit for comparison with the indicator items in the medical quality indicator system. The business indicator library refers to a structured database stored within the medical quality indicator system of each clinical business system. This database includes all quality indicator items in each clinical business scenario, and each indicator item is associated with corresponding business type, scope of application, data standard and other attribute information. The indicator item list refers to the indicators retrieved and organized from the business indicator library. The item list is categorized and arranged according to clinical business types. Each item in the list contains core information such as indicator item code, indicator item name, and indicator item definition. The list is in a standardized tabular format and can be directly used for item-by-item comparison with the clinical feature item set. The indicator item list is retrieved and organized from the business indicator database. This list is categorized and arranged according to clinical business types. Each item in the list contains core information such as indicator item code, indicator item name, and indicator item definition. The associated indicator item group refers to the set of mutually matching feature items and indicator items selected after item-by-item comparison between the clinical feature item set and the indicator item list. Each combination in this set has a clear matching relationship label and is the core component for constructing the mapping indicator set.

[0037] Furthermore, as another embodiment of the present invention, the step of generating a visual form framework for each clinical business system based on the key indicator items in the mapping indicator set includes: parsing the clinical records and indicator ranges corresponding to the key indicator items in the mapping indicator set; configuring the chart types and data display areas in the preset form according to the clinical records and indicator ranges to form a preliminary framework outline; and generating a visual form framework for each clinical business system based on the preliminary framework outline and the form specifications of each clinical business system.

[0038] The clinical record refers to text or data directly related to the key indicators in the mapping indicator set, recording patient diagnosis and treatment information. It includes details of the corresponding diagnosis and treatment operations, patient status descriptions, and testing instructions. This record originates from the diagnosis and treatment databases of various clinical business systems. The indicator range refers to the reasonable value interval or content boundary corresponding to the key indicators in the mapping indicator set, defined by clinical diagnosis and treatment guidelines, industry standards, and the management requirements of various clinical business systems. It includes information such as the normal reference range, abnormal thresholds, and data accuracy requirements for the indicators. The preset form refers to a set of standardized form templates pre-stored in the system, adapted to different clinical business scenarios. It includes basic form layout structures, general data items, and expandable display modules. The chart type refers to the chart style used to visually present the key indicator data, covering various forms such as line charts, bar charts, tables, and dashboards. Different chart types are adapted to different scenarios. Different types of indicator data, such as time-series data suitable for line charts and comparative data suitable for bar charts, must be selected to match the display requirements of clinical records and indicator ranges. The data display area refers to the area specifically designated in the preset form template for presenting data of specific key indicator items. This area has clear boundaries and location, and includes sub-areas such as data display area, indicator name labeling area, and auxiliary explanation area. The size and layout of the area must be adapted to the display requirements of the corresponding chart type. The preliminary framework outline refers to the basic form prototype formed after configuring the chart type and data display area according to the clinical records and indicator ranges. It includes core elements such as the arrangement order of key indicator items, the distribution position of each data display area, and the selected chart type identifier. The form specifications refer to the set of standardized requirements for forms formulated by each clinical business system, covering the form layout style, data display format, font specifications, identification rules, content integrity requirements, etc.

[0039] S4. Extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form.

[0040] The embodiments of the present invention can integrate core medical information scattered in medical records, so that the form content and the framework structure are accurately matched, and the key medical elements filled in can enrich the core content in the framework and clarify the medical basis corresponding to each item in the form.

[0041] The key diagnostic and treatment elements refer to information units extracted from hospital medical records that reflect the core situation of the patient's diagnosis and treatment. These elements include the patient's chief complaint, diagnosis, treatment plan, key points of medical orders, and key results of physical examinations. Each element has a clear diagnostic and treatment stage association identifier and data type attribute, and is the basic information unit that constitutes the core content of the form. The preset form item template refers to a standardized item configuration template that is pre-stored in the system and adapted to the visual form frame. It includes elements such as the name, data filling format, content specifications, and arrangement order of each item in the form. The template is set according to different clinical business scenarios and can directly guide the accurate filling of key diagnostic and treatment elements. The "completely filled form" refers to the form formed after the extracted key diagnostic and treatment elements are completely filled into the visual form frame according to the requirements of the preset form item template. This form includes the entire structure of the frame and the diagnostic and treatment information corresponding to each item. There are no missing items, and the filled content conforms to the item specifications. It is a form that has been initially improved and is waiting for further updates and calibrations.

[0042] In detail, the extraction of key medical elements from the in-hospital medical records can be achieved by using a method based on medical domain rules and named entity recognition models. The in-hospital medical record text is input into a medical NER model based on BERT fine-tuning to identify and label core entities such as diseases, drugs, and medical procedures. Finally, the labeled entities are matched and verified with the rule base, and the structured key medical elements are obtained by classification and organization.

[0043] As an embodiment of the present invention, the step of filling the diagnostic and treatment elements into the visual form frame according to a preset form item template to obtain a fully filled form includes: obtaining the item name sequence and corresponding content format requirements in the preset form item template; extracting specific content from the key diagnostic and treatment elements whose semantic information matches the item name sequence; standardizing the specific content according to the content format requirements, and then loading the standardized content, combined with the semantic information, into the corresponding item position in the visual form frame; and generating a fully filled form after confirming that the content embedded in the corresponding item position carries the key diagnostic and treatment elements.

[0044] The item name sequence refers to a set of names stored in a preset form item template, arranged in the order of form items. This sequence includes all item names in the visual form frame, with each name corresponding to a key diagnostic element type. The order of the sequence is consistent with the item layout of the visual form frame. The content format requirements refer to the standardized specifications formulated by the preset form item template for the content to be filled in for each item name, covering text expression specifications, numerical unit specifications, content length limits, data precision requirements, etc. The semantic information refers to the attribute information carried by the key diagnostic element itself, used to characterize the element category and core meaning. This information includes the diagnostic stage label to which the element belongs and the theme of the element content, and is the core basis for judging whether the key diagnostic element matches the item name sequence. The specific content refers to the content extracted from the key diagnostic element. The content elements that match the semantic information of the corresponding entries in the entry name sequence are the original diagnostic and treatment information that has not been formatted and includes various forms such as text descriptions, numerical records, and operation instructions. This content is the basic material for filling the visual form frame. The corresponding entry position refers to the specific area in the visual form frame where the entry that matches the semantic information of the key diagnostic and treatment elements is located. This position has clear coordinates and boundary ranges. Each position corresponds to a unique entry name and corresponds one-to-one with the order of entries in the preset form entry template. This is the designated area for loading the standardized content. The implanted content refers to the final content that can be directly loaded into the visual form frame after the specific content has been formatted according to the content format requirements. This content conforms to the form display specifications of various clinical business systems and includes standardized text expressions, unified numerical units, and compliant content length.

[0045] S5. Based on the diagnostic voice data, update the medical and nursing information entries in the fully filled form, and combine them with the preset multimodal large model to output a medical and nursing intelligent form that conforms to each clinical business system, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0046] The embodiments of the present invention can supplement the image-related diagnosis and treatment information in the form, improve the core content of the form, and synchronize the intelligent medical and nursing form to the medical and nursing work terminal, which can open up the key link of data transmission, allowing medical and nursing staff to directly obtain the integrated diagnosis and treatment information, and provide comprehensive information support for clinical work.

[0047] The medical and nursing information items refer to specific items set in the fully populated form to carry diagnostic and treatment information related to the diagnostic voice data. These items correspond to the imaging diagnostic conclusions, imaging feature descriptions, and examination site descriptions in clinical diagnosis and treatment. The medical and nursing intelligent form refers to a standardized form that is generated after updating the medical and nursing information items in the fully populated form based on the diagnostic voice data and after optimization processing by a preset multimodal large model. This form is adapted to the requirements of various clinical business systems. It integrates multi-source diagnostic and treatment information such as vital sign test data, in-hospital medical records, and diagnostic voice data, and has a structured data format and a visual display form, which can be directly used in clinical diagnosis and treatment scenarios. The medical and nursing work terminal refers to the terminal equipment used by medical and nursing staff to carry out their daily diagnosis and treatment work, including medical and nursing workstation computers, mobile nursing terminals, and doctor ward round tablets. This terminal establishes a stable data transmission link with various clinical business systems, has the functions of receiving, viewing, and storing forms, and is the target carrier for the synchronous distribution of the medical and nursing intelligent form.

[0048] As an embodiment of the present invention, updating the medical information entries in the fully filled form based on the diagnostic voice data includes: identifying the voice sign description in the diagnostic voice data and converting the voice sign description into an audit guide list associated with the form entries in the fully filled form; verifying the entry record information in the audit guide list item by item; when the entry record information differs from the voice sign description, marking the entry record information as pending correction; and updating the medical information entries in the fully filled form in the pending correction state.

[0049] The medical sign description refers to the characteristic information extracted from diagnostic voice data that can characterize the pathological or physiological state of a patient's body tissues or organs, covering lesion location, size, density characteristics, boundary clarity, and association with surrounding tissues. The review guide list refers to the guide list associated with the completed form entries after the voice sign description is converted according to preset mapping rules. This list includes the unique identifier of each form entry corresponding to the review guide, the sign matching standard, and the key points for information verification, serving as a reference for verifying the record information item by item. The entry record information refers to the existing original record content of the medical and nursing information entries in the completed form. This content includes preliminary diagnosis and treatment information related to the diagnostic voice data, a summary of examination conclusions, and a brief description of imaging features, serving as the object for consistency comparison with the voice sign description. Each piece of information corresponds to a specific entry position in the form. The pending correction status refers to the status marker marked on the entry record information when there is inconsistency or deviation between the entry record information and the content of the voice sign description. This status includes a description of the deviation type, a summary of the difference content, and a correction priority mark, used to clearly indicate that the corresponding medical and nursing information entry needs to be updated.

[0050] Furthermore, as another embodiment of the present invention, the review guide list is a standardized checklist generated by converting the speech feature description in the diagnostic speech data and associated with specific items in the fully filled form. This list can map the image language description in the image to structured review items for form entries, providing a clear target list for item-by-item verification.

[0051] In detail, the process involves combining a pre-defined multimodal large model to output intelligent medical and nursing forms that conform to the requirements of each clinical business system. These intelligent forms are then synchronized to the medical and nursing work terminals. This can be achieved by real-time collection and updating of fully filled structured form data and unstructured diagnostic voice data. Both types of data are pre-processed, converting the form data into model-readable feature tensors and extracting image feature vectors from the diagnostic voice data using a convolutional neural network. Subsequently, these two types of features are input into the pre-defined multimodal large model, which calls upon the pre-stored form specification knowledge base of each clinical business system within the model to perform format calibration, information completion, and logical verification of the form content. This ensures that the form fields, data format, and content structure fully match the requirements of the corresponding clinical business system, generating standardized intelligent medical and nursing forms. Next, a form synchronization transmission channel is established, employing an encrypted transmission method based on a medical data transmission protocol. The intelligent medical and nursing forms are categorized and labeled according to their terminal affiliation. Finally, the labeled forms are pushed to the corresponding medical and nursing work terminals via the medical business intranet.

[0052] Specifically, as a key embodiment of the present invention, when the medical and nursing work terminal is in a poor environment and continues to input voice, it specifically includes: (1) Deployment of local cache module: A lightweight encrypted cache module is built into mobile medical terminals, such as ward round tablets and mobile APPs. The cache capacity is set to 1GB and is specifically used to store voice data when the network is interrupted. The AES-128 encryption algorithm is used to ensure data security and avoid privacy leakage. (2) Speech segmentation strategy: Set dynamic segmentation threshold and adaptively adjust the segment length according to the speech input rate to ensure that the semantics of each segment is relatively complete, while reducing the data volume of a single segment to facilitate subsequent transmission; The file name format of each segment is "Patient ID-Device ID-Timestamp-Segment Sequence Number" to ensure uniqueness; (3) Network status monitoring: A dual monitoring mechanism is adopted. On the one hand, the network connection status (Wi-Fi / cellular network) is obtained through the mobile device system API. On the other hand, the network connectivity is detected through the server heartbeat packet. When no heartbeat response is received for 3 consecutive times or the network connection status is marked as "disconnected", it is determined that the network is poor. (4) Optimization of interrupted transmission: An incremental upload strategy is adopted, uploading only the missing audio segments instead of all data; an audio segment index is established on the server side and stored according to patient ID for easy query and verification; if the network is interrupted again during the transmission process, the upload is stopped immediately and the local cache status is updated, and the transmission is resumed when the network is restored to ensure that no data is lost. (5) Semantic coherence repair: After the server receives the complete voice segment, it uses the semantic repair module of the multimodal big model to complete and repair the semantics at the interruption point by combining the contextual semantic state, such as keywords in the uploaded segment and incomplete sentences; for example, if the sentence before the interruption is "patient's body temperature is 38.5℃, accompanied by cough", and the uploaded segment after the interruption is "no sputum, respiratory rate 22 breaths / min", the semantic repair module will automatically complete it to "patient's body temperature is 38.5℃, accompanied by cough, no sputum, respiratory rate 22 breaths / min", ensuring the semantic coherence of the speech-to-text result; (6) Status recovery mechanism: The mobile device records the operation status of voice input, such as the position of the input interface and the selected form items. When the network is interrupted and then resumed, it will automatically restore the operation interface and input status before the interruption. Medical staff can continue to complete voice input without having to reoperate, thus improving the user experience.

[0053] Compared to the problems described in the background art, the embodiments of the present invention can integrate patient diagnosis and treatment information scattered across different business systems. Furthermore, the multimodal data encompassing vital sign examination data, in-hospital medical records, and diagnostic voice data can comprehensively present various key states during the diagnosis and treatment process. Next, the embodiments of the present invention can integrate the scattered vital sign examination data, analyze the dynamic changes in vital sign indicators during the patient's visit, and clearly present the evolution of each vital sign indicator during the patient's visit, providing structured data support for the generation of intelligent medical forms. Then, after mapping the diagnosis and treatment feature vectors to the medical quality indicator systems of each clinical business system, the embodiments of the present invention can connect the diagnosis and treatment feature vectors with… Clinical quality standards are established to form a set of mapping indicators with clear clinical orientation, thereby clarifying the core content of forms under various clinical business systems. Furthermore, embodiments of this invention can integrate core diagnostic and treatment information scattered in medical records, ensuring precise matching between form content and framework structure. The key diagnostic and treatment elements filled in enrich the core content of the framework, clarifying the diagnostic and treatment basis corresponding to each item in the form. Furthermore, embodiments of this invention can supplement image-related diagnostic and treatment information in the form, improving the core content of the form. The intelligent medical and nursing form is synchronized to the medical and nursing work terminal, enabling the key link in data transmission to be opened up, allowing medical staff to directly obtain the integrated diagnostic and treatment information, providing comprehensive information support for clinical work. Therefore, this invention achieves effective fusion of multimodal medical data, and the generated form can completely reflect the overall picture of diagnosis and treatment and the dynamic evolution of vital signs, improving clinical work efficiency and medical quality.

[0054] like Figure 3 The diagram shown is a functional module diagram of a medical and nursing intelligent form generation system based on a multimodal large model according to the present invention.

[0055] The medical and nursing intelligent form generation system 300 based on a multimodal large model described in this invention can be installed in an electronic device. Depending on the functions implemented, the medical and nursing intelligent form generation system based on a multimodal large model includes a multimodal data module 301, a feature vector module 302, a form framework module 303, a form filling module 304, and an intelligent form module 305. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0056] In this embodiment of the invention, the functions of each module / unit are as follows: The multimodal data module 301 is used to acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes vital sign test data, in-hospital medical records and diagnostic voice data; The feature vector module 302 is used to generate a sequence of changes in the vital signs of the medical staff and patients during the consultation process by combining the preset multimodal large model with the vital sign test data, and to construct a diagnostic feature vector with the sequence of changes in vital signs as the core. The form framework module 303 is used to map the diagnosis and treatment feature vector to the medical quality indicator system of each clinical business system to obtain a mapping indicator set, and generate a visual form framework under each clinical business system based on the key indicator items in the mapping indicator set. The form filling module 304 is used to extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form. The intelligent form module 305 is used to update the medical and nursing information entries in the fully filled form according to the diagnostic voice data, and output the medical and nursing intelligent form that conforms to the clinical business system by combining the preset multimodal large model, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0057] In detail, the modules in the medical and nursing intelligent form generation system 300 based on a multimodal large model described in this embodiment of the invention adopt the same approach as described above when in use. Figure 1 The method uses the same technique as the one described above for generating intelligent medical forms based on a multimodal large model, and can produce the same technical effect, so it will not be elaborated here.

[0058] In one embodiment, a computer device is provided, which may be a server or a client, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements functions or steps on the server or client side of a multimodal large-scale medical intelligent form generation method.

[0059] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data; Using a pre-defined multimodal large model combined with the vital sign test data, a sequence of vital sign changes of the medical staff and patients during the consultation process is generated, and a diagnostic feature vector with the sequence of vital sign changes as the core is constructed. After mapping the diagnostic feature vectors to the medical quality indicator system of each clinical business system, a mapping indicator set is obtained, and a visual form framework under each clinical business system is generated based on the key indicator items in the mapping indicator set. Extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form; Based on the diagnostic voice data, update the medical and nursing information entries in the fully filled form, and combine them with the preset multimodal large model to output a medical and nursing intelligent form that conforms to each clinical business system, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0060] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data; Using a pre-defined multimodal large model combined with the vital sign test data, a sequence of vital sign changes of the medical staff and patients during the consultation process is generated, and a diagnostic feature vector with the sequence of vital sign changes as the core is constructed. After mapping the diagnostic feature vectors to the medical quality indicator system of each clinical business system, a mapping indicator set is obtained, and a visual form framework under each clinical business system is generated based on the key indicator items in the mapping indicator set. Extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form; Based on the diagnostic voice data, update the medical and nursing information entries in the fully filled form, and combine them with the preset multimodal large model to output a medical and nursing intelligent form that conforms to each clinical business system, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

[0061] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0062] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct RAM (RDRAM), direct memory bus RAM (DRDRAM), and memory bus RAM (RDRAM), etc.

[0063] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0064] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0065] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for generating intelligent medical and nursing forms based on a multimodal large model, characterized in that, The method includes: Acquire multimodal data of medical staff and patients in various clinical business systems, wherein the multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data; Using a pre-defined multimodal large model combined with the vital sign test data, a sequence of vital sign changes of the medical staff and patients during the consultation process is generated, and a diagnostic feature vector with the sequence of vital sign changes as the core is constructed. After mapping the diagnostic feature vectors to the medical quality indicator system of each clinical business system, a mapping indicator set is obtained, and a visual form framework under each clinical business system is generated based on the key indicator items in the mapping indicator set. Extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form; Based on the diagnostic voice data, update the medical and nursing information entries in the fully filled form, and combine them with the preset multimodal large model to output a medical and nursing intelligent form that conforms to each clinical business system, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

2. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 1, characterized in that, The process of generating a sequence of vital sign changes during the medical visit by combining a preset multimodal large model with the vital sign test data includes: Generate an initial set of vital signs in the continuous detection process of the vital sign test data, and extract time-series vital sign pairs containing the vital sign collection time and specific vital sign values ​​from the initial set of vital signs; After analyzing the trend segments of the vital signs corresponding to the values ​​of the vital signs in the time series, the time intervals associated with the trend segments of the vital signs are connected to obtain the segments of changes in vital signs. The numerical deviation records in the segments of the changes in vital signs are corrected, and based on the corrected segments, a sequence of changes in vital signs of the medical staff and patients during the consultation process is constructed.

3. The method for generating intelligent medical forms based on a multimodal large model as described in claim 2, characterized in that: The aforementioned vital sign change segment represents a continuous time data unit formed by identifying the changing trend of a specific physiological indicator from continuously detected vital sign data. It consists of several temporally adjacent and trend-consistent vital sign measurements and their corresponding time points, representing the relatively complete physiological state evolution process of the patient within a certain period of time.

4. The method for generating intelligent medical forms based on a multimodal large model as described in claim 1, characterized in that, The acquisition of multimodal data on medical staff and patients in various clinical business systems includes: After parsing the patient identifiers of the medical staff and patients in each clinical business system, a secure data link is established corresponding to the access node under each clinical business system. Collect multi-mode data streams from the secure data link and perform format alignment on the multi-mode data streams; The aligned data stream is aggregated according to the timestamp of the patient identifier to obtain multimodal data.

5. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 1, characterized in that, After mapping the diagnostic feature vectors to the medical quality indicator systems of each clinical business system, a set of mapped indicators is obtained, including: Analyze the clinical feature itemset in the diagnostic feature vector; Based on the clinical feature item set, the medical quality indicator system in each clinical business system is invoked to obtain the indicator item list of the business indicator library under the medical quality indicator system. After comparing the set of clinical features with the list of indicator items item by item, a group of associated indicator items is obtained, and the associated indicator items group is mapped into the mapping indicator set in the medical quality indicator system.

6. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 1, characterized in that, The step of generating a visual form framework for each clinical business system based on key indicator items in the mapping indicator set includes: The clinical records and indicator ranges corresponding to the key indicator items in the mapping indicator set are analyzed. Based on the clinical records and indicator ranges, configure the chart types and data display areas in the preset forms to form a preliminary framework outline; Based on the preliminary framework outline and the form specifications of each clinical business system, a visual form framework for each clinical business system is generated.

7. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 1, characterized in that, The step of filling the diagnostic and treatment elements into the visual form frame according to the preset form item template to obtain a fully filled form includes: Retrieve the sequence of item names and corresponding content format requirements from the preset form item template; Extract the specific content from the key diagnostic and treatment elements that matches the sequence of entry names; After standardizing the specific content according to the content format requirements, the standardized content, combined with the semantic information, is loaded into the corresponding item position in the visual form frame. After confirming that the content embedded in the corresponding entry position carries the key diagnostic and treatment elements, a fully populated form is generated.

8. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 1, characterized in that, The step of updating the medical information entries in the fully populated form based on the diagnostic voice data includes: Identify the speech feature descriptions in the diagnostic speech data and convert the speech feature descriptions into an audit guide list associated with the form entries in the fully filled form; Verify the record information of each item in the audit guidelines list; When there is a discrepancy between the entry record information and the speech feature description, the entry record information is marked as being in a state to be corrected. Update the healthcare information entries in the fully populated form under the state to be corrected.

9. The method for generating intelligent medical and nursing forms based on a multimodal large model as described in claim 8, characterized in that, The audit guide list is a standardized checklist generated by converting speech feature descriptions from diagnostic speech data and associating them with specific items in a fully filled form. This list maps image language descriptions in images to structured audit items for form entries, providing a clear target list for item-by-item verification.

10. A medical and nursing intelligent form generation system based on a multimodal large model, characterized in that, The system includes: The multimodal data module is used to acquire multimodal data of medical staff and patients in various clinical business systems. The multimodal data includes physical examination data, in-hospital medical records and diagnostic voice data. The feature vector module is used to generate a sequence of changes in the vital signs of the medical staff and patients during the consultation process by combining the preset multimodal large model with the vital sign test data, and to construct a diagnostic feature vector with the sequence of changes in vital signs as the core. The form framework module is used to map the diagnosis and treatment feature vectors to the medical quality indicator system of each clinical business system to obtain the mapping indicator set, and generate a visual form framework under each clinical business system based on the key indicator items in the mapping indicator set. The form filling module is used to extract key medical elements from the in-hospital medical records and fill the medical elements into the visual form frame according to the preset form item template to obtain a fully filled form. The intelligent form module is used to update the medical and nursing information entries in the fully filled form based on the diagnostic voice data, and output the medical and nursing intelligent form that conforms to the clinical business systems by combining the preset multimodal large model, and synchronize the medical and nursing intelligent form to the medical and nursing work terminal.

Images