Newborn serious illness management system and system based on AI large model, and electronic equipment

Abstract

The invention discloses a neonatal severe illness management system and system based on an AI large model and electronic equipment, and relates to the technical field of artificial intelligence and health cross, and the system comprises a data access integration module, a model training module and a personalized health suggestion generation module; the data access integration module is used for acquiring multi-modal medical data and performing cross-source heterogeneous fusion processing on the multi-modal medical data to generate a data set; the model training module is used for training a preset large language model based on the data set to obtain a neonatal severe health management model; and the personalized health suggestion generation module is used for generating a corresponding personalized health management scheme according to the vital sign data of the target newborn by using the newborn severe health management model. According to the invention, a health management scheme accurately fitting the individual condition can be provided for each newborn, and the effect and quality of newborn intensive care management are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and health technology, and in particular to a neonatal critical care management system, system and electronic device based on an AI large model. Background Technology

[0002] In the medical field, neonatal intensive care management has always been a challenging problem. With the continuous advancement of medical technology, the requirements for neonatal intensive care management are becoming increasingly stringent. Most existing neonatal intensive care management systems have limitations and struggle to meet the complex clinical needs of today. In terms of data processing, traditional systems can only acquire medical data from a single source, such as simple physiological indicators, and cannot effectively perform cross-source heterogeneous fusion processing of multimodal medical data, including but not limited to imaging data, laboratory test results, and real-time vital sign monitoring data. This results in low data utilization and an inability to form a complete and effective dataset to support accurate analysis and decision-making. Regarding model construction and training, previous model training methods were relatively simple, failing to fully utilize the powerful capabilities of pre-built large language models and unable to incorporate the rich nodes and relationships in the knowledge graph constructed from the neonatal specialty knowledge base as constraints into the model training process, making it difficult to achieve knowledge-driven intelligent management. Furthermore, the application of retrieval-enhanced generation mechanisms is insufficient, failing to efficiently retrieve target information from the knowledge graph and combine it with the large language model to generate health management plans tailored to individual characteristics. As for the development of personalized health management plans, traditional systems are often based on general medical guidelines or experience, lacking personalized consideration of the vital signs data of the target newborns. They cannot provide each newborn with a precise health management plan that fits their individual situation, which limits the effectiveness of the management. Summary of the Invention

[0003] The technical problem to be solved by this invention is to address the shortcomings of existing technologies. Specifically, it provides a neonatal critical care management system, system, and electronic device based on an AI large-scale model, as detailed below: 1) In a first aspect, the present invention provides a neonatal critical care management system based on an AI large-scale model, the specific technical solution of which is as follows: It includes: a data access and integration module, a model training module, and a personalized health suggestion generation module; The data access and integration module is used to: acquire multimodal medical data, and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; The model training module is used to: train a pre-set large language model based on a dataset to obtain a neonatal critical care health management model. During the training process, nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints to be input into the pre-set large language model, and / or, using a retrieval enhancement generation mechanism, target information is retrieved from the knowledge graph, and then combined with the pre-set large language model to generate personalized health management plans. The personalized health recommendation generation module is used to generate corresponding personalized health management plans based on the vital sign data of the target newborn and using the neonatal critical care health management model.

[0004] The beneficial effects of the newborn critical care management system based on an AI large model provided by this invention are as follows: At the data processing level, the data access and integration module can acquire multimodal medical data and perform cross-source heterogeneous fusion processing to generate datasets. This effectively solves the problem that traditional systems can only acquire medical data from a single source and cannot fuse multimodal data, greatly improving data utilization and laying a solid foundation for subsequent accurate analysis and decision-making. Regarding model building and training, the model training module trains a pre-set large language model based on the dataset. By using nodes and relationships in the knowledge graph as constraints, it fully utilizes the powerful capabilities of the pre-set large language model, achieving knowledge-driven intelligent management and overcoming the limitations of previous simple model training methods. Simultaneously, utilizing a retrieval-enhanced generation mechanism, it can efficiently retrieve target information from the knowledge graph and combine it with the large language model to generate personalized health management plans, compensating for the shortcomings of traditional systems in applying retrieval-enhanced generation mechanisms. In the formulation of personalized health management plans, the personalized health suggestion generation module generates plans based on the target newborn's vital sign data and the neonatal intensive care health management model. This avoids the drawbacks of traditional systems that rely on general guidelines or experience and lack personalized considerations, providing each newborn with a precise health management plan tailored to their individual situation, significantly improving the effectiveness and quality of neonatal intensive care management.

[0005] Based on the above solution, the newborn critical care management system based on an AI large model of the present invention can be further improved as follows.

[0006] Furthermore, by leveraging a retrieval-enhanced generation mechanism, target information is retrieved from a knowledge graph and then combined with a pre-defined large language model to generate personalized health management plans, including: Based on data from the dataset, target information is retrieved from the knowledge graph, including clinical guidelines and pathological features; The target information is aligned and corrected with the inference output of the preset large language model to generate a personalized health management plan.

[0007] The beneficial effects of adopting the above-mentioned further approach are as follows: Accurate retrieval of target information from the knowledge graph based on the data in the dataset fully leverages the rich knowledge in the neonatal specialist knowledge base, integrating key information such as clinical guidelines and pathological characteristics into the generation process of health management plans. This overcomes the shortcomings of traditional systems in insufficiently applying retrieval enhancement generation mechanisms. Aligning and correcting the retrieved target information with the inference output of the preset large language model effectively improves the accuracy and reliability of personalized health management plans. This ensures that the generated plans not only conform to authoritative knowledge such as clinical guidelines but also accurately match the individual pathological characteristics of the target newborns. It compensates for the shortcomings of traditional systems in organically combining knowledge graph information with large language models, thus providing more targeted and scientifically based personalized health management plans for neonatal intensive care, improving management effectiveness and decision-making quality.

[0008] Furthermore, it also includes a remote collaboration module, which is used for: The neonatal critical care health management model can be deployed to different terminals, or the neonatal critical care health management model can be lightweighted and then deployed to different terminals to enable real-time response to remote requests and push of personalized health management plans.

[0009] The beneficial effects of adopting the above-mentioned further solutions are as follows: Firstly, deploying the neonatal critical care health management model to different terminals, or deploying it after lightweight processing, enables the model to adapt to various device environments, expanding its application scope. Secondly, this deployment method enables real-time response to remote requests, timely delivery of personalized health management plans, breaking geographical limitations and enhancing the sharing and accessibility of medical resources. Simultaneously, the efficient operation of the lightweight model on terminals not only reduces the performance requirements of terminal devices but also ensures the consistency and stability of the model across different terminals, guaranteeing the accuracy and reliability of the generated health management plans. Furthermore, this remote collaboration mechanism facilitates multi-terminal data synchronization and updates, further optimizing model training and improvement, enhancing the overall intelligence level of the neonatal critical care management system, and providing newborns with more timely and efficient health management services.

[0010] Furthermore, multimodal medical data includes: real-time vital signs data from monitoring devices, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

[0011] The beneficial effects of adopting the above-mentioned further approach are: it provides comprehensive and rich multi-dimensional data support for the neonatal intensive care health management model. This makes the dataset more complete and detailed, fully reflecting the health status of newborns, providing a solid foundation for model training, and thus improving the model's accuracy and reliability. Simultaneously, the fusion of multimodal data helps enhance the model's comprehensive analytical capabilities regarding neonatal health status, making the generated personalized health management plans more accurate and comprehensive, fully meeting the individualized needs of different newborns, and effectively improving the effectiveness and quality of neonatal intensive care management.

[0012] 2) Secondly, the present invention also provides a method for neonatal intensive care management based on an AI large model, the specific technical solution of which is as follows: Acquire multimodal medical data and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; Based on the dataset, a pre-set large language model is trained to obtain a neonatal critical care health management model. During the training process, nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints and input into the pre-set large language model. And / or, the target information is retrieved from the knowledge graph using a retrieval enhancement generation mechanism, and then combined with the pre-set large language model to generate a personalized health management plan. Based on the vital signs data of the target newborns and using the neonatal critical care health management model, a corresponding personalized health management plan is generated.

[0013] Based on the above scheme, the newborn critical care management method based on an AI large model of the present invention can be further improved as follows.

[0014] Furthermore, by leveraging a retrieval-enhanced generation mechanism, target information is retrieved from a knowledge graph and then combined with a pre-defined large language model to generate personalized health management plans, including: Based on data from the dataset, target information is retrieved from the knowledge graph, including clinical guidelines and pathological features; The target information is aligned and corrected with the inference output of the preset large language model to generate a personalized health management plan.

[0015] Furthermore, it also includes: The neonatal critical care health management model can be deployed to different terminals, or the neonatal critical care health management model can be lightweighted and then deployed to different terminals to enable real-time response to remote requests and push of personalized health management plans.

[0016] Furthermore, multimodal medical data includes: real-time vital signs data from monitoring devices, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

[0017] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the electronic device implements any of the above-mentioned methods for neonatal critical care management based on AI large models.

[0018] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for neonatal critical care management based on an AI large model.

[0019] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is a schematic diagram of the structure of a neonatal critical care management system based on an AI large model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a method for a neonatal critical care management system based on an AI large model, according to an embodiment of the present invention. Detailed Implementation

[0021] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0022] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0023] like Figure 1 As shown in the figure, an AI-based large-scale neonatal critical care management system according to an embodiment of the present invention includes: a data access and integration module, a model training module, and a personalized health suggestion generation module; The data access and integration module is used to: acquire multimodal medical data, and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; Multimodal medical data includes: real-time vital signs data from monitoring devices, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

[0024] The cross-source heterogeneous fusion processing of multimodal medical data is achieved in the following ways: 1) First implementation method: ① Multimodal medical data, including real-time vital signs, laboratory test results, medical images, and text medical records, are acquired from monitoring equipment, laboratory information systems, imaging systems, and electronic medical record systems. This data is collected and integrated through a data access and integration module. All data sources are transmitted to the data processing center in a unified format. Monitoring equipment data is transmitted in time-series format, including indicators such as heart rate and respiratory rate; laboratory test results are recorded in structured tables; imaging system data is compressed and encoded before transmission as medical images; and the electronic medical record system provides text-based medical record records.

[0025] ② The acquired data is cleaned, denoised, and standardized. For real-time vital signs data, filtering algorithms are used to remove noise points; missing values ​​are filled and outliers are corrected for test index data; grayscale standardization and size uniformity are performed on medical images; and text medical records are processed through word segmentation, part-of-speech tagging, and stop word removal to transform them into text features suitable for analysis.

[0026] ③ Utilize multimodal feature extraction algorithms to extract key features from various types of data. Extract time-series features from real-time vital signs data, statistical features from diagnostic indicators, image features from medical images using convolutional neural networks, and textual features from medical records using natural language processing techniques. Then, employ fusion algorithms, such as attention-based fusion models, to weight and fuse the features from each modality. The weights are dynamically adjusted based on the contribution of each modality to the newborn's health status, achieving deep fusion of cross-source heterogeneous data and forming a unified feature representation.

[0027] ④ Combine the fused feature data with corresponding label data (such as neonatal disease diagnosis results, treatment records, etc.) to construct a dataset for model training. The dataset is stored in a database or file system according to a specific format for easy access and management by the subsequent model training module. During data storage, ensure data security and privacy by employing encryption and other technologies to protect sensitive information.

[0028] 2) The second implementation method: ① Vital sign streaming data (including ECG, blood oxygen, and blood pressure waveforms) from monitoring devices are collected in real time via a medical device interface gateway. Simultaneously, structured laboratory test indicators are obtained by calling the Laboratory Information System (LIS) API, DICOM format medical image sequences are extracted from the Picture Archiving System (PACS), and unstructured text medical records are retrieved from the Electronic Medical Record System (EMR) via the FHIR protocol. Vital sign data, test indicators, medical images, and text medical records are transmitted to a distributed storage cluster via an encrypted channel. A raw data warehouse is established by generating unique data identifiers based on patient ID and timestamp. Specifically: A connection is established with the monitoring equipment via a medical device interface gateway. Following appropriate communication protocols such as HL7, real-time streaming data of vital signs, including ECG, blood oxygen, and blood pressure waveforms, is collected. The interface gateway performs preliminary data processing, such as format verification and unit conversion, to ensure data accuracy and consistency, and then temporarily stores it in a local buffer. Simultaneously, the Laboratory Information System (LIS) API is invoked, sending data requests to the LIS based on preset patient identifiers and other parameters. Upon receiving the request, the LIS retrieves and returns the corresponding structured laboratory indicator data. The interface gateway receives the indicator data and performs necessary parsing and encapsulation to ensure compatibility with the vital signs data in terms of format and structure, facilitating subsequent integration and processing. DICOM format medical image sequences are extracted from the Picture Archiving System (PACS). Communication with the PACS is established via the DICOM protocol, using key information such as patient ID to locate and retrieve the required image sequences. The interface gateway performs integrity verification on the acquired DICOM image data to ensure its accuracy and completeness, and extracts and organizes its metadata for subsequent association and storage with other data. Unstructured text medical records are retrieved from the Electronic Medical Record System (EMR) according to the FHIR protocol. According to the FHIR resource definition, a corresponding query request is constructed, and the patient ID is sent to the EMR system as the query condition. After the EMR system returns the text medical record data related to the patient, the interface gateway performs necessary cleaning and preprocessing on the data, removing redundant information and non-standard formatting, thereby improving the quality and readability of the text medical record data. The collected, acquired, extracted, and retrieved vital sign data, laboratory indicators, medical images, and text medical record data are transmitted through a pre-established encrypted channel. Encryption protocols such as SSL / TLS are used to encrypt the data, ensuring its security and confidentiality during transmission and preventing theft or tampering. After the data is transmitted to the distributed storage cluster, a unique data identifier is generated based on the combination of patient ID and timestamp. This identifier is then used to store the data in the original data warehouse, achieving orderly storage and rapid retrieval of the data, providing a reliable data foundation for subsequent medical data analysis, clinical decision support, and other applications.

[0029] ② Based on patient ID, vital sign data, laboratory test results, medical images, and text medical records are linked to a unified timeline: millisecond-level resampling and alignment are performed on vital sign data; acquisition time labels are added to laboratory test results; time-domain matching is achieved based on the DICOM header file StudyDateTime of medical images; and key event timestamps are parsed in text medical records using a clinical event timestamp extractor. Spatially, isotropic resampling and N4 bias field correction are performed on medical images, Z-score normalization is applied to vital sign data, laboratory test results are converted to LOINC standard encoding, and text medical records are mapped to entities using UMLS medical ontology.

[0030] For vital sign data, a patient ID-based index was used to extract it from the original data warehouse. Given the potential for variations in the frequency of vital sign data collection, resampling functions in programming languages ​​(such as the `resample()` function in Python's pandas library) were employed to uniformly resample the data to millisecond-level time intervals. When setting resampling rules, methods such as linear interpolation or spline interpolation were used to estimate the data values ​​at missing time points, ensuring data continuity and alignment on the timeline. After alignment, a uniform and accurate timestamp was generated for each vital sign data point, serving as the basis for subsequent unified timeline analysis.

[0031] For structured laboratory test data, identification matching is first performed based on patient ID. Since test indicators are typically derived from samples collected at specific time points, the raw data may lack precise collection time labels. Therefore, a rule-based interpolation model is constructed. Using known time reference points such as patient admission time and test request time as a basis, and combining the typical turnaround time (TAT) of the test items, reasonable collection time labels are added for each test indicator according to the collection order and logical relationship. For example, for blood routine test indicators, given their typically short TAT, the collection time is estimated to be close to the test request time; while for culture-related test indicators, considering the longer culture period, the collection time label is reasonably delayed. The added time labels are refined to the minute level to allow for initial correlation with vital sign data in the time dimension.

[0032] Specifically, for medical imaging data, the focus is on analyzing the StudyDateTime field in the DICOM header file, which records the start time of the imaging examination. The StudyDateTime value is extracted based on the patient ID. Simultaneously, considering the duration of the imaging sequence and combining it with the timeline of vital sign data, vital sign data segments within the imaging examination time period are extracted. By calculating the overlap between the imaging examination time period and the vital sign data timeline, a time interval mapping algorithm is used to match the imaging data and vital sign data in the time domain, providing a temporal correspondence for subsequent comprehensive analysis.

[0033] The text medical record event time parsing process involves using a clinical event timestamp extractor (a model developed based on natural language processing technology, such as using regular expressions to match common time formats and combining medical terminology to identify time-related information described in the context) for unstructured text medical records. This extractor uses the patient ID as a clue to locate key clinical event descriptions in the text (such as symptom onset time, treatment implementation time, etc.) and parses out the corresponding timestamps. The extracted timestamps are also accurate to the minute. These key time points are then compared with the timelines of vital signs data, laboratory indicators, and imaging data to construct a temporal mapping between key events in the text medical record and multi-source data, thus completing the construction of a unified timeline.

[0034] The process involves reading DICOM format medical image data into an image processing framework (such as SimpleITK or the Pydicom library in Python). First, based on the spatial resolution information of the image, it is determined whether isotropic resampling is necessary. If the voxel spacing varies significantly in different directions, a uniform target value for voxel spacing (e.g., 0.5mm × 0.5mm × 0.5mm) is set. Then, using methods such as cubic spline interpolation or nearest neighbor interpolation, the image is resampled to make the voxel spacing in the three dimensions more consistent, achieving isotropy. Next, to address the N4 bias field effect (common in MRI images), an N4 bias field correction algorithm (a bias field estimation and elimination method based on a multinomial model) is applied. By estimating the smooth bias field component in the image and subtracting it from the original image, the grayscale uniformity of the image is improved, providing higher-quality image data for subsequent image analysis (such as lesion identification and registration). For various indicators in vital sign data (such as heart rate, blood pressure, and blood oxygen saturation), Z-score normalization is required due to their varying numerical ranges. Based on historical data for each vital sign indicator, its mean μ and standard deviation σ are calculated. For newly acquired vital sign data points x, the distribution is transformed using the formula (x-μ) / σ, converting the data distribution of each indicator into a standard normal distribution with a mean of 0 and a standard deviation of 1. Normalized vital sign data eliminates dimensional differences, facilitating subsequent data fusion analysis and model training. It also helps avoid visual biases caused by significant differences in numerical ranges between different indicators during visualization.

[0035] For test indicator data, based on an existing mapping table (a pre-established lookup table of local test items and LOINC standard codes), the local name or code of each test indicator is converted into its corresponding LOINC standard code. This conversion process ensures the semantic standardization and interoperability of test indicators, facilitating data integration and analysis with other medical institutions. Simultaneously, for medical entities in text medical records (such as disease names, symptom descriptions, and drug names), the metadata resources and mapping tools provided by UMLS (Unified Medical Language System) (such as querying the standard conceptual identifiers and semantic types of entities through the UMLS Metathesaurus database) are used to standardize the mapping of medical entities in the text. In specific implementations, methods such as string matching and semantic similarity calculation (e.g., using UMLS concept unique identifiers and semantic networks to calculate the semantic relevance between entities) can be used to convert non-standardized entities in the text into standard concepts in the UMLS ontology, thereby achieving semantic standardization of text medical records and laying the foundation for cross-system and cross-data source medical knowledge fusion and analysis.

[0036] ③ A parallel feature extraction network is constructed to process standardized data. Specifically, vital sign data is input into a temporal convolutional network (TCN) to extract multi-scale waveform features; test indicators are input into a fully connected autoencoder to generate low-dimensional embedding vectors; medical images are extracted using a 3D ResNet-50 to extract multi-level visual features; and text medical records are generated into diagnostic semantic vectors using the BioClinicalBERT model. The outputs of each branch are processed by a feature selection module to retain Top-K discriminative features, forming four sets of dimension-aligned feature tensors.

[0037] The standardized and time-aligned vital sign data is input into a Temporal Convolutional Network (TCN). The TCN consists of multiple one-dimensional convolutional layers, with the kernel size and number of channels for each layer pre-set according to the data characteristics. During convolution, dilated convolution is used to expand the receptive field and capture the features of vital sign waveforms at different temporal scales. By stacking multiple TCN layers, multi-scale waveform features are extracted progressively, ultimately yielding the feature tensor output of this branch, with a shape of (batch size, number of channels, time step).

[0038] The test index data, converted to LOINC standard encoding, is input into a fully connected autoencoder. The autoencoder consists of an encoder and a decoder. The encoder progressively compresses the input test index vector into a low-dimensional space through several fully connected layers, forming low-dimensional embedding vectors. The decoder then uses fully connected layers to restore the low-dimensional embedding vectors back to vectors close to the original dimensions to achieve the autoencoding function. Through training, the reconstruction error is minimized, thereby extracting representative low-dimensional embedding vectors as the feature tensor of the test index, with dimensions of (batch size, low-dimensional embedding dimension).

[0039] For medical images that have undergone isotropic resampling and N4 bias field correction, the 3D ResNet-50 model is used for processing. 3D ResNet-50 extends the convolutional kernels to three dimensions on top of the traditional two-dimensional residual network (ResNet) to adapt to the three-dimensional structure of medical images. Image data is input through convolutional layers and multiple three-dimensional residual modules (each module contains several three-dimensional convolutions, batch normalization, activation functions, and residual connection operations), and then global average pooling is performed to obtain a feature tensor of multi-level visual features, with a shape of (batch size, number of feature channels).

[0040] Specifically, text medical records, after entity standardization mapping using the UMLS medical ontology, are used to generate semantic vectors using the BioClinicalBERT model. BioClinicalBERT is a medical domain-specific language model pre-trained and fine-tuned based on the BERT architecture. The text medical records are segmented and input into the model. The model outputs the semantic vector for each word based on word embedding layers, multi-layer self-attention mechanisms, and a feedforward neural network. The vector corresponding to the [CLS] label is taken as the overall diagnostic semantic vector of the text medical record, i.e., the feature tensor of this branch, with dimensions of (batch size, semantic vector dimension).

[0041] After the feature tensors generated by each branch are output, they enter the feature selection module. This module uses feature selection algorithms based on statistical methods such as mutual information or chi-square tests to calculate the correlation between each feature dimension and the target clinical diagnostic label, and selects the Top-K feature dimensions with high discriminative power from each set of feature tensors. By indexing the data of the corresponding dimensions of the feature tensors of each branch, four sets of dimension-aligned feature tensors are formed, providing a concise and highly discriminative feature representation for subsequent fusion analysis or model training, ensuring that subsequent processing can efficiently utilize key feature information for accurate medical data mining and diagnostic prediction.

[0042] ④ A three-level fusion architecture is designed to process feature tensors. Specifically, the primary fusion uses a cross-modal attention gate to calculate the affinity matrix between the visual features of medical images and the semantic vectors of text medical records; the intermediate fusion uses a time synchronization module to dynamically associate the temporal features of vital signs data with test indicators; the final fusion layer introduces a graph neural network (GNN), which uses patient clinical events as nodes and multimodal feature relationships as edges, and aggregates the global context through a graph attention mechanism to output a fusion vector with unified dimensions as the dataset.

[0043] Among them, multi-level visual feature tensors (dimensions such as...) are obtained from medical image feature extraction networks. ,in Indicates batch size. (representing the number of visual feature channels), obtain the diagnostic semantic vector tensor (dimensions such as...) from the text medical record semantic vector generation module. , The semantic vector dimension is represented by a key vector, which is then fed into a cross-modal attention gate. This gate consists of two fully connected layers and an activation function: first, linear transformations are applied to the visual features and the text semantic vector to obtain the query vector and key vector, both with dimension [not specified]. ,in Let be the hyperparameters for the attention dimension; then calculate their dot product and divide by . The initial affinity matrix is ​​obtained, and then it is normalized to the affinity matrix using the softmax function. .

[0044] Among them, the time-series characteristics of vital signs data (dimensions such as...) , Indicates the time step. (representing the normalized dimensions of vital signs) and the low-dimensional embedding vector of the test index (dimensions such as...) The input time synchronization module first uses linear interpolation to align the time axis of vital signs with the time labels of the test indicators, generating a vital signs tensor under the synchronized time axis. Dynamic correlation vector of test indicators ,in The time step is set after synchronization; then, through a temporal attention mechanism, the dynamic correlation vector of the test index focuses on the relevant parts of the vital signs at each time step, resulting in the fused intermediate feature tensor. This enables dynamic correlation between the two modalities in the time dimension.

[0045] Among them, medical image-text feature nodes (node ​​feature dimensions such as) are collected after primary fusion affinity matrix association. Key event nodes in the intermediate-level fusion vital signs-test indicator feature sequence (feature dimensions such as...) ) and the original patient's clinical events (such as admission, surgery, discharge, etc., feature dimensions) As nodes in a graph neural network (GNN), construct the node feature tensor. ,in Indicates the total number of nodes. This represents the dimension after concatenating features from nodes of different sources. Edges are constructed based on the similarity between multimodal features (such as the cosine similarity between medical image visual features and text semantic vectors, and the Pearson correlation between vital sign features and test indicator vectors), and an adjacency matrix is ​​built. The node features and adjacency matrix are input into the GNN. Its graph attention layer (GAT) calculates the attention weights between nodes through a graph attention mechanism, aggregating the features of neighboring nodes to update its own features. After multiple GAT layers, a global average pooling layer is used to aggregate the node features into a single fusion vector. ,in To ultimately fuse vector dimensions and achieve global context aggregation, a fused vector with unified dimensions is output as a dataset for subsequent analysis.

[0046] 3) The third implementation method: ① Based on preset clinical event triggers, vital sign data from monitoring devices are captured in real time via medical IoT middleware. This triggers the Laboratory Information System (LIS) API to retrieve structured test indicators, links the Picture Archiving System (PACS) to retrieve DICOM format medical images, and activates the Electronic Medical Record (EMR) interface to extract text medical records. Vital sign data, test indicators, medical images, and text medical records are transmitted to edge computing nodes via HL7 message queues, generating a spatiotemporal correlation index based on clinical event IDs.

[0047] Among these, a series of clinical event triggers were defined collaboratively by the medical team and data scientists, covering scenarios such as abnormal fluctuations in neonatal vital signs (e.g., tachycardia / bradycardia, sudden drop in blood oxygen saturation), threshold breaches of specific laboratory indicators (e.g., abnormal blood glucose levels, abnormal white blood cell count in routine blood tests), and key findings from imaging examinations (e.g., suspected intraventricular hemorrhage, abnormal lung translucency suggesting acute respiratory distress syndrome). These triggers are deployed in the form of a rule engine within the medical IoT middleware, continuously monitoring the data stream from monitoring devices.

[0048] The medical IoT middleware establishes connections with various medical information systems via secure, certified APIs. When a trigger is activated, the middleware captures the vital signs data stream generated by monitoring equipment in real time. Simultaneously, it calls the Laboratory Information System (LIS) API to obtain structured laboratory indicators related to the clinical event, retrieves corresponding medical images through the DICOM Web API of the Picture Archiving System (PACS), and activates the FHIR interface of the Electronic Medical Record System (EMR) to extract relevant textual medical record information. This process ensures the timeliness and accuracy of data capture, and all operations comply with stringent requirements for medical data security and privacy protection. Vital signs data are collected in a combination of continuous waveforms and discrete values, encompassing dynamic information such as electrocardiogram waveforms and periodic measurements of non-invasive blood pressure, reflecting the real-time physiological state of newborns. Laboratory indicators are presented in structured tabular form, containing detailed information such as specific numerical results, reference ranges, and units, such as key parameters like pH and partial pressure of carbon dioxide in blood gas analysis. Medical image data is stored in DICOM format, including image pixel data, patient identification information, and examination equipment parameters, and is accessed and transmitted across systems via DICOM Web Services. The text medical record information includes unstructured text such as doctors' diagnosis and treatment records and nursing observation notes. Natural language processing technology is used to perform preliminary entity recognition and relation extraction, laying the foundation for subsequent data fusion.

[0049] The captured data is encapsulated into a message queue according to the HL7 standard format and transmitted to edge computing nodes near the data source to reduce network latency and perform initial data processing. At the edge computing nodes, the data first undergoes cleaning and preprocessing, removing noisy data, filling in missing values, and resampling vital sign data to standardize time frequencies. Then, an indexing service generates spatiotemporal correlation indexes for various data types based on clinical event IDs, ensuring that heterogeneous data related to the same clinical event can be quickly located and correlated, providing efficient data access support for subsequent comprehensive analysis. The entire process, driven by a high-performance data processing engine, ensures the real-time nature and accuracy of the data, meeting the stringent requirements for rapid response in neonatal intensive care.

[0050] ② The waveform features of vital signs data are mapped to UMLS ontology concept nodes, and the test indicators are bound to knowledge graph entities through LOINC encoding. The lesion features of medical images are associated with RadiLex terms, and the text medical records are aligned to SNOMED CT codes through entity links. All modal data are converted to RDF triple format to form a patient-centered semantic network.

[0051] The process involves preprocessing vital sign data, including filtering, noise removal, and feature extraction. Signal processing techniques such as wavelet transform are used to extract key feature points and morphological features of the waveforms. The MetaMap tool provided by UMLS (Unified Medical Language System) is used to match the extracted features with concepts in the UMLS ontology. By calculating the semantic similarity between the feature descriptions and ontology concepts, the best-matching UMLS concept node is found, thus mapping the waveform features to the corresponding ontology concepts.

[0052] The test indicator data is typically in a structured format, containing specific test items and numerical results. A unique LOINC code is assigned to each test indicator using the LOINC (Laboratory Observation Identification Code) coding system. A mapping table is then established to bind the test indicator's name, unit, and other information to its LOINC code. These binding relationships are then stored in a knowledge graph, serving as entity nodes associated with the test indicators, thus achieving the binding of test indicators to knowledge graph entities.

[0053] In medical imaging, lesion feature identification primarily relies on image analysis algorithms. Computer vision techniques, such as convolutional neural networks (CNNs), are used to detect lesions and extract features from DICOM-formatted medical images. Then, based on the RadiLex (radiological lexicon) standard terminology set, a mapping relationship between lesion features and RadiLex terms is established. Through pattern matching and semantic similarity calculation, the extracted lesion features are associated with the most appropriate RadiLex terms, ensuring a standardized description of image features.

[0054] Entity recognition and linking within the text medical record is a crucial step. Natural language processing techniques, such as Named Entity Recognition (NER), are used to identify entities related to diseases, symptoms, and treatments from the text. The identified entities are then matched and linked with SNOMED CT (Systematic Nomenclature for Clinical Terminology) terms using APIs or tools provided by SNOMED CT. By calculating the semantic similarity and contextual relevance between the text entities and SNOMED CT terms, the best-matching SNOMED CT code is determined, achieving alignment between the text medical record and the standardized code.

[0055] After standardizing and mapping the data across the various modalities, all data is converted to RDF (Resource Description Framework) triple format. Each triple consists of a subject, a predicate, and an object. The subject is typically the patient or a related medical entity, the predicate represents an attribute or relation, and the object is a specific value or associated entity. Using an RDF library (such as RDFlib), the modal data is integrated into a patient-centric graph database (such as Neo4j) to form a semantic network. In this network, the patient acts as the central node, connected to various medical data nodes. Relationship edges represent the connections between data, thereby achieving comprehensive and semantic integration and representation of patient medical information, providing a solid foundation for the generation of personalized health management plans.

[0056] ③ Vital signs data and test indicators form positive sample pairs, and joint embedding is learned through temporal contrastive loss; medical images and text medical records are optimized in the embedding space after cross-modal attention alignment; the four types of modal data are generated into modality-specific embedding vectors through graph convolutional networks (GCN) on the semantic network, and a feature matrix with uniform output dimension is output.

[0057] For vital signs data and laboratory test results, the first step is to align them temporally to ensure comparability within the same time window. Then, treating vital signs and test results as positive sample pairs, their joint embeddings are learned using a temporal contrastive loss function. Specifically, a deep neural network maps vital signs and test results to the same embedding space. The temporal contrastive loss function encourages embedding vectors to be close between positive sample pairs within the same time window, while embedding vectors to be far apart between negative sample pairs from different time windows or different patients. This captures the temporal correlation between vital signs and test results, improving the model's sensitivity to changes in patient condition.

[0058] Medical images and text medical records originate from different modalities, necessitating cross-modal alignment. First, a pre-trained convolutional neural network is used to extract feature vectors from the medical images, while a natural language processing model is used to extract semantic vectors from the text medical records. Then, a cross-modal attention mechanism is constructed to allow image and text features to mutually influence each other. By calculating their attention scores, feature weights are dynamically adjusted to achieve better alignment between image and text features in the embedding space. Finally, the aligned image and text features are fused to optimize the embedding space, enabling it to more accurately represent the patient's clinical condition.

[0059] The four modalities of data (vital signs, laboratory indicators, medical images, and text medical records) processed above are integrated into a constructed semantic network. In the network, each node represents a data point or concept, and edges represent the relationships between them. A Graph Convolutional Network (GCN) is used to process this graph-structured data. GCN updates the embedding vector of each node by aggregating information from neighboring nodes, thereby capturing the complex relationships and structural information between the data. For each modality's data node, GCN generates a corresponding modality-specific embedding vector. These vectors contain specific information about that modality's data within the semantic network and are comparable to the embedding vectors of other modalities in the same vector space. The modality-specific embedding vectors generated from the four modalities are then processed to unify their dimensions. Through linear transformations or other feature mapping techniques, the embedding vectors of different modalities are mapped to the same output dimension, forming a unified feature matrix. This feature matrix serves as the model's output, providing a comprehensive, multimodal feature representation for subsequent diagnostic, treatment decisions, or other medical tasks. This unified feature matrix can comprehensively reflect the patient's condition, improving the accuracy and reliability of medical decisions.

[0060] ④ Using clinical event time points as graph nodes and node attributes as row vectors of the feature matrix, a gated graph neural network (GGNN) is used to perform the following: a) update homogeneous features using edge convolution within a modality; b) calculate the interaction between medical image nodes and text medical record nodes using multi-head graph attention across modalities; c) dynamically expand the graph structure and recalibrate edge weights when adding new clinical events, and output the fused multidimensional tensor as the dataset.

[0061] A dynamic graph structure is constructed using clinical event time points as graph nodes. Each node represents a clinical event at a specific time point, and its attributes are initialized as row vectors of a feature matrix. This vector integrates embedded representations of vital signs data, laboratory indicators, medical images, and textual medical records at that time point. The graph structure is stored in a graph database, supporting real-time querying and updating.

[0062] In Gated Graph Neural Networks (GGNNs), for each modality (such as vital signs or diagnostic indicators), edge convolution operations are used to update the features of the current node. Edge convolution updates the features of the current node by aggregating the features of neighboring nodes, as shown in the following formula: in, It is a node In time eigenvectors, It is a node The set of neighboring nodes, and It is a learnable weight matrix. It is an activation function. Through edge convolution operations, the model can capture feature interactions at different time points within the same modality.

[0063] In GGNN, a multi-head graph attention mechanism is used to compute the interaction between medical image nodes and text medical record nodes. Multi-head graph attention calculates attention scores between nodes independently using multiple attention heads, and then concatenates or averages the results, as shown in the following formula: in, , and It is a learnable weight matrix. This is the feature dimension. Through multi-head graph attention, the model is able to capture the complex interactions between medical images and text medical records.

[0064] When a new clinical event is added, the graph structure is dynamically expanded. New nodes are added to the graph, and their node attributes are initialized to the corresponding feature vectors. Simultaneously, based on the relationship between the new node and existing nodes, edge weights are recalculated through edge convolution and multi-head graph attention to ensure the connectivity of the graph structure and the accuracy of feature representation. The dynamic expansion and edge weight calibration process is as follows: in, It is the node feature matrix of the current graph. It is the initial feature vector of the new node. It is a function used to detect the relationship between a new node and existing nodes.

[0065] After processing by GGNN, the fused multidimensional tensor output is used as a dataset for subsequent training and prediction of the neonatal intensive care management model. The output multidimensional tensor contains a comprehensive feature representation of each clinical event time point, which can comprehensively reflect the neonatal's health status and changes in condition.

[0066] The model training module is used to: train a pre-set large language model based on a dataset to obtain a neonatal critical care health management model. During the training process, nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints to be input into the pre-set large language model, and / or, using a retrieval enhancement generation mechanism, target information is retrieved from the knowledge graph, and then combined with the pre-set large language model to generate personalized health management plans. Specifically, the construction of the knowledge graph based on the neonatal specialty knowledge base is achieved in the following ways: ① Knowledge is extracted from the neonatal specialist knowledge base, specifically from authoritative textbooks, clinical guidelines, medical literature, and expert experience in the field of neonatology. The extracted knowledge covers key information such as disease diagnostic criteria, treatment plans, drug use guidelines, and reference values ​​for examinations and tests.

[0067] ② Utilize natural language processing technology to analyze the acquired knowledge text and identify key entities, such as disease names, symptoms, examination items, and treatment drugs. Simultaneously, employ dependency parsing and pattern matching methods to extract relationships between entities, such as "disease-symptom" relationships, "disease-treatment plan" relationships, and "drug-indication" relationships.

[0068] ③ Construct a knowledge graph based on a graph database (such as Neo4j). Identified entities are treated as nodes in the knowledge graph, and relationships are treated as edges between nodes. Each node contains detailed information about the entity; for example, a disease node contains attributes such as disease name, cause, and pathogenesis. Edges contain information such as the type and weight of the relationship. The nodes and relationships in the knowledge graph are as follows: Nodes include disease nodes (such as premature infant respiratory distress syndrome, neonatal jaundice), symptom nodes (such as rapid breathing, jaundice), examination and testing nodes (such as complete blood count, liver function tests), treatment plan nodes (such as phototherapy, mechanical ventilation), and drug nodes (such as vitamin K, antibiotics). Each node contains rich attribute information, such as the etiology, pathogenesis, and epidemiology of the disease node.

[0069] Relationships include "disease-symptom" relationships (e.g., premature infant respiratory distress syndrome - tachypnea), "disease-examination / test" relationships (e.g., neonatal jaundice - blood bilirubin test), "disease-treatment plan" relationships (e.g., premature infant respiratory distress syndrome - mechanical ventilation), "treatment plan-medication" relationships (e.g., phototherapy - blue light therapy device), and "medication-indication" relationships (e.g., vitamin K - prevention of neonatal hemorrhage). These relationships reflect the intrinsic connections between different entities within the neonatal specialty, providing rich semantic constraints for model training and improving the accuracy and reliability of the model.

[0070] ④ Invite neonatologists to validate the constructed knowledge graph to ensure the accuracy and completeness of the knowledge. Simultaneously, regularly update the knowledge graph with the latest research findings and clinical practice to maintain its timeliness and effectiveness, meeting the ever-changing needs of neonatal intensive care management.

[0071] The specific implementation process of inputting the nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base as constraints into the preset large language model is as follows: ① Represent the nodes and relationships in the knowledge graph in a formal way. Nodes are represented as sets. Each node This corresponds to an entity in the knowledge graph (such as disease, symptom, examination and testing items, treatment plan, etc.), along with its attribute information (such as disease name, etiology, pathogenesis, etc.). The relationship is represented as a set. , where each relation It indicates the relationship between two nodes (such as "disease-symptom" relationship, "disease-treatment plan" relationship, etc.), and includes information such as the type and weight of the relationship.

[0072] ② Graph embedding techniques (such as TransE, TransR, etc.) are used to map nodes and relations in the knowledge graph to a low-dimensional vector space. Specifically, for each node... and relationships Learn a low-dimensional vector representation and These vectors capture the semantic information of nodes and relationships, as well as the structural relationships between them. During graph embedding, structural constraints in the knowledge graph are preserved by optimizing the objective function (such as a distance-based scoring function). For example, in the TransE model, the objective is to minimize triples. The distance, i.e. This allows the relevant nodes and relationships to have a reasonable geometric relationship in the vector space.

[0073] ③ During the training of the pre-defined large language model, the embedding vectors of the knowledge graph are fused as additional input features. Specifically, in the model's input layer or encoder layer, the word embeddings corresponding to the text input are concatenated or weighted and summed with the embedding vectors of relevant nodes and relationships in the knowledge graph. For example, for a disease name mentioned in the input text, the corresponding node embedding vector is retrieved from the knowledge graph and concatenated with the text word embedding vector before being input into the large language model. In this way, when processing text information, the model can simultaneously utilize the rich structured knowledge in the knowledge graph as constraints, enhancing its understanding and reasoning ability regarding text semantics.

[0074] ④ Design an attention mechanism to enable the pre-defined large language model to focus on knowledge graph nodes and relationships relevant to the current task when generating output. Specifically, during the model's decoding phase, dynamically calculate the attention score for each node and relationship in the knowledge graph based on the current decoding state. For example, this can be achieved by calculating the dot product of the decoder's hidden state and the knowledge graph node embedding vectors, or by using an attention network. Then, use the softmax function to convert these scores into a probability distribution, which serves as weights for different nodes and relationships in the knowledge graph. These weights are used to weight and sum the embedding vectors of knowledge graph nodes and relationships, generating a context vector containing knowledge constraints. This context vector is then combined with the decoder's state to generate the next output word.

[0075] Through the above process, the nodes and relationships in the knowledge graph are effectively input into the pre-set large language model as constraints, enabling the model to make full use of the structured knowledge in the neonatal specialty knowledge base during training and inference, thereby improving the performance and accuracy of the model in neonatal critical care health management tasks.

[0076] This includes utilizing a retrieval-enhanced generation mechanism to retrieve target information from a knowledge graph, and then combining this with a pre-defined large language model to generate personalized health management plans, including: ① Based on data from the dataset, retrieve target information from the knowledge graph. The target information includes clinical guidelines and pathological features.

[0077] The process involves preprocessing multimodal medical data from the dataset. For real-time vital signs data, time series analysis is used to extract key features, such as the mean, variability, and trend of heart rate. For laboratory test indicators, standardization is performed and outlier indicators are extracted. For medical images, convolutional neural networks (CNNs) are used to extract image features. For text medical records, natural language processing techniques are employed for word segmentation, part-of-speech tagging, and entity recognition to extract key information such as disease names, symptoms, and treatment plans. The extracted feature vectors are then associated with the corresponding newborn identification information to form a structured feature dataset, providing input for subsequent retrieval processes.

[0078] The process involves constructing search requests based on clinical needs. For example, if a personalized health management plan needs to be developed for a newborn, the search request might include the newborn's abnormal vital signs, abnormal laboratory test results, and the name of the preliminarily diagnosed disease. The search request is then converted into a vector representation. A pre-trained encoder (such as BERT) encodes the text portion of the search request, generating a text feature vector. For non-text features (such as vital sign feature vectors and test result feature vectors), the extracted feature vectors are used directly. Feature vectors from different modalities are concatenated or fused (e.g., using a weighted fusion mechanism with attention) to form a unified search request vector.

[0079] This process involves indexing nodes and relationships within the knowledge graph. Using the indexing capabilities of graph databases (such as Neo4j's indexing mechanism), indexes are built for nodes (e.g., disease nodes, symptom nodes, treatment plan nodes) and relationships (e.g., "disease-symptom" relationships, "disease-treatment plan" relationships) in the knowledge graph to facilitate rapid retrieval. Based on the retrieval request vector, a search is performed within the knowledge graph. Vector similarity-based retrieval methods (e.g., cosine similarity or inner product similarity) are employed to calculate the similarity between the retrieval request vector and the embedding vectors of each node and relationship in the knowledge graph. Simultaneously, combining the structural information of the knowledge graph (e.g., node types, relationship paths), graph traversal algorithms (e.g., breadth-first search or depth-first search) are used to expand the search along relationship edges from the initial node (the node that best matches the retrieval request), searching for clinical guidelines and pathological features relevant to the retrieval request.

[0080] The retrieved target information is validated. It is compared with authoritative sources in the knowledge graph (such as clinical guideline publishers and authoritative medical literature), and the search results are ranked based on relevance and confidence. A machine learning ranking algorithm (such as a gradient boosting tree-based ranking model) is used to comprehensively consider factors such as the matching degree between the search request and the target information, the credibility of the target information source, and the centrality of the target information in the knowledge graph. This ranking aims to provide the most relevant and reliable information for subsequent health management plan generation, ensuring the accuracy and reliability of the retrieved clinical guideline and pathological feature information.

[0081] Through the above process, clinical guidelines and pathological features related to neonatal critical care health management can be retrieved efficiently and accurately from the knowledge graph, providing strong support for the generation of personalized health management plans.

[0082] ② Align and correct the target information with the inference output of the preset large language model to generate a personalized health management plan, specifically: Clinical guidelines and pathological features retrieved from the knowledge graph are stored in a structured manner, ensuring that each information item has a clear identifier and attributes, such as the issuing organization of the guideline, its scope of application, and a detailed description of the pathological features. A pre-defined large language model generates preliminary health management plan text based on the input vital sign data and medical record information, including diagnostic suggestions, treatment measures, and nursing points. Natural language processing is applied to the preliminary inference output of the pre-defined large language model to extract key concepts and features, such as disease names, treatment plans, and drug recommendations. Entity recognition and relation extraction techniques are used to match these features with the target information retrieved from the knowledge graph, finding corresponding nodes and relationships. Semantic similarity calculation methods (such as cosine similarity) are used to compare the semantic similarity between the key concepts in the inference output and the target information. For parts that do not match or have low similarity, adjustments are made according to the target information. For example, if the treatment plan in the inference output is inconsistent with the clinical guidelines in the target information, the recommended plan in the guidelines is adopted first. Clinical guidelines and pathological features in the target information are used as constraints and integrated into the generation process of the large language model. The specific method involves dynamically adjusting the probability distribution of generated words during the model's decoding phase to make the generated content more aligned with the target information. Simultaneously, an attention mechanism is used to enhance the model's focus on key nodes and relationships within the target information. Combining the target newborn's vital sign data and medical records, machine learning algorithms are used to personalize and optimize the adjusted health management plan. For example, medication dosage or treatment intensity is adjusted based on the newborn's real-time vital sign data; specific nursing interventions are customized based on pathological characteristics. Neonatologists are invited to validate and evaluate the generated personalized health management plan to ensure its scientific validity and feasibility. Feedback from doctors and users is collected to further optimize the model and knowledge graph. The validated and optimized personalized health management plan is integrated into a complete text document, formatted according to standard medical document formats to ensure clarity and organization. The personalized health management plan is presented to healthcare professionals and parents through a user-friendly interface (such as a mobile application or website) for easy understanding and implementation. The interface design emphasizes the visual presentation of information, such as using charts to display vital sign trends and medication schedules.

[0083] During the training of the pre-defined large language model, real-time vital sign data is acquired from monitoring equipment, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems. This multimodal medical data undergoes cleaning, noise reduction, and standardization. Multimodal feature extraction algorithms are used to extract key features from each data type. Then, an attention-based fusion model is employed to weightedly fuse these features, forming a unified feature representation and generating a dataset for model training. Entities and relationships are extracted from a neonatal specialist knowledge base. Natural language processing techniques are used for entity recognition and relationship extraction to construct a knowledge graph. Nodes in the knowledge graph include diseases, symptoms, examination and test items, and treatment plans, while relationships include "disease-symptom" and "disease-treatment plan." Graph embedding techniques (such as TransE) are used to map the nodes and relationships in the knowledge graph to a low-dimensional vector space, learning the embedding vector for each node and relationship. During the training of the pre-defined large language model, the embedding vectors from the knowledge graph are concatenated or weightedly summed with the word embedding vectors of the text input as the model input. Simultaneously, an attention mechanism is designed to ensure the model focuses on task-related knowledge graph nodes and relationships when generating output. A retrieval request is constructed based on the dataset, and a pre-trained encoder encodes the text portion to generate a retrieval request vector. A vector similarity-based retrieval is performed in the knowledge graph, combined with a graph traversal algorithm, starting from the initial node to find clinical guidelines and pathological features relevant to the retrieval request. The retrieved target information is validated to ensure its accuracy and reliability, and ranked according to relevance and confidence. A pre-set large language model generates a preliminary health management plan based on the input vital sign data and medical record information. The retrieved target information is aligned and corrected with the preliminary plan. Key concepts in the preliminary plan are extracted, matched and semantically compared with the target information, and adjustments are made to any mismatches. Simultaneously, the target information is incorporated as a constraint into the model's decoding process, dynamically adjusting the probability distribution of generated words. Combined with specific data from newborns, machine learning algorithms are used to personalize and optimize the plan, and specialist physicians are invited to validate and evaluate it to ensure the plan's scientific validity and feasibility.

[0084] The preset large language model can be GPT-3.5, GPT-4, or BERT, or other large language models can be selected according to the actual situation.

[0085] Personalized health management plans can include dietary plans, exercise plans, or disease prevention plans, specifically: ① Based on the newborn's health status, growth and development stage, and special needs, personalized dietary plans are generated, including breastfeeding recommendations, formula selection, and the timing and types of complementary food introduction. During training of the pre-defined large-scale language model, a dataset containing vital signs data such as newborn weight, height, and blood routine tests, as well as information on breast milk composition, formula nutritional components, and complementary food nutritional components, is collected. Successful and unsuccessful dietary cases are also included, noting adverse reactions such as malnutrition and allergies. This data is converted into text descriptions (e.g., "Weight 2.5kg, Hemoglobin 130g / L, breastfeeding recommended, 8-10 times daily") and input into the pre-defined large-scale language model. Supervised learning is used to teach the mapping relationship between the input and the dietary recommendation output. Hyperparameters are adjusted during training to optimize the loss function, improving the accuracy and applicability of the generated recommendations. Pediatric nutrition information from a knowledge graph constrains the model output. For example, if a newborn's weight gain is slow, the model generates a high-calorie breastfeeding plan, which is then verified and corrected using the knowledge graph to ensure the recommendation is scientifically sound.

[0086] ② Based on the newborn's developmental indicators and health status, develop personalized health and exercise plans, covering passive exercises, massage, swimming, and recommendations for exercise types, intensity, and frequency. When training the pre-set large language model, integrate the newborn's muscle tone, motor development milestones, and other vital signs data, as well as case studies and effect evaluations of different exercise interventions, marking adverse reactions such as over-exercise and under-exercise. Convert the data into text (e.g., "Normal muscle tone, 3 months old, recommended passive exercises twice daily, 15 minutes each time") and input it into the pre-set large language model. Use supervised learning; the model learns the mapping between input and exercise suggestion output, continuously adjusting hyperparameters and optimizing the loss function during training. Combine the knowledge of children's motor development in the knowledge graph to constrain the model output. For example, if a newborn has low muscle tone, the model generates a passive exercise plan with increased intensity, retrieves the "low muscle tone - exercise plan" relationship in the knowledge graph to correct it, and generates a gradual plan to ensure safety and effectiveness.

[0087] ③ Personalized disease prevention plans are provided based on the individual characteristics and susceptibility of newborns, including vaccination schedules, daily care guidelines, and environmental adjustment suggestions. When training the pre-defined large language model, data on newborns' basic physical signs, family medical history, and living environment are integrated. Vaccination cases and their effects, common neonatal disease causative factors and preventive measures are collected, and adverse reactions such as disease occurrence are labeled. The data is converted into text descriptions and input into the pre-defined large language model. Supervised learning is used to allow the model to learn the mapping relationship between the input data and disease prevention suggestions, continuously optimizing hyperparameters and the loss function during training. The model output is constrained by neonatal disease prevention knowledge from the knowledge graph. For example, if a newborn has a family history of allergies, when generating a vaccination plan, the model searches the "allergy history - vaccine selection" relationship in the knowledge graph to avoid allergen vaccines and strengthen observation suggestions, making the plan more targeted and safer.

[0088] The personalized health recommendation generation module is used to generate corresponding personalized health management plans based on the vital sign data of the target newborn and using a neonatal intensive care health management model. Specifically: The data access and integration module collects multimodal medical data from target newborns in real time, including real-time vital signs data from monitoring devices, laboratory test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems. This data undergoes cross-source heterogeneous fusion processing to generate a complete dataset. This dataset is then input into a neonatal critical care health management model. This model is based on a pre-defined large language model and incorporates a knowledge graph constructed from a neonatal specialist knowledge base. It uses a retrieval-enhanced generation mechanism to retrieve target information such as clinical guidelines and pathological characteristics relevant to the newborn's condition from the knowledge graph. Next, the retrieved target information is aligned and corrected with the inference output of the large language model to generate a preliminary personalized health management plan. Finally, the intelligent diagnosis and decision support module, as well as the intelligent early warning and intervention module, validate and optimize the plan to ensure its scientific validity and feasibility before pushing it to medical staff and parents, achieving precise health management for newborns.

[0089] Optionally, the above technical solution also includes a remote collaboration module, which is used for: The neonatal critical care health management model can be deployed to different terminals, or the neonatal critical care health management model can be lightweighted and then deployed to different terminals to enable real-time response to remote requests and push of personalized health management plans.

[0090] The specific implementation process of lightweighting the neonatal critical care health management model is as follows: First, parametric pruning techniques are used to remove redundant connections and neurons from the model. Weight parameters with minimal impact on the model output are selected by setting thresholds and set to zero, thereby reducing the number of parameters and simplifying the model structure. Then, quantization is performed, converting high-precision parameters (such as 32-bit floating-point numbers) into low-precision parameters (such as 8-bit integers), reducing storage requirements and accelerating computation. Next, knowledge distillation is employed, using the original model as the teacher model and its output to guide the training of small student models. This allows the student models to learn the key knowledge and feature representations of the teacher model, further reducing the model size. This process ensures the model's ability to process multimodal information such as neonatal vital signs data, while also enabling efficient deployment on different terminals, meeting the needs for real-time response to remote requests and personalized health management plan delivery.

[0091] When personalized health management plans are dietary plans, the terminals include: mobile applications, smart refrigerators, and wearable devices. Parents or guardians can receive dietary advice, ingredient lists, and feeding plans through a dedicated application on their smartphones or tablets. A smart refrigerator connected to the home network can suggest food storage and usage, and provide meal recommendations based on the health management plan. Wearable devices, such as smartwatches, can remind parents of feeding times and provide real-time dietary suggestions.

[0092] When the personalized health management plan includes a health and exercise program, the terminals are: smart fitness trackers, tablets, smart TVs, or smart strollers. Smart fitness trackers are wearable devices designed for infants that monitor movement activity and send feedback to parents' mobile devices. Tablets and smart TVs provide interactive video guidance on infant movement, instructing parents to perform passive exercises or massages. Smart strollers are equipped with sensors to monitor the infant's movement and activity levels and provide exercise suggestions to parents via an app.

[0093] When the personalized health management plan is a disease prevention plan, the terminals include: a mobile health application, a smart thermometer, health monitoring devices, and the hospital or clinic's management system. The mobile health application sends vaccination reminders, daily care suggestions, and environmental adjustment prompts to parents. The smart thermometer and health monitoring devices connect to the mobile device to continuously monitor the infant's health indicators and provide preventative measures according to the plan. The hospital and clinic's management system, through a secure network connection, allows medical professionals to remotely access and adjust the infant's health management plan.

[0094] In another embodiment, the present invention provides a neonatal critical care management system based on an AI large-scale model, comprising a data access and integration module, an AI large-scale model core inference module, an intelligent screening module, an intelligent diagnosis and decision support (CDSS) module, an intelligent early warning and intervention module, a discharge management and follow-up module, and a primary care adaptation and remote collaboration module. The data access and integration module is used to: connect to various NICU monitoring equipment, laboratory information systems (LIS), imaging systems (PACS), electronic medical record systems (EMR), etc., to collect and integrate multimodal medical data. The AI ​​large-scale model core inference module is used to: integrate a large language model (LLM) with a multimodal model, embed a neonatal specialist knowledge base, and possess semantic understanding capabilities. The system includes: reasoning and personalized recommendation capabilities; an intelligent screening module for automatically analyzing data such as birth defects, high-risk medical history, and genetic metabolic screening to achieve early risk warning; an intelligent diagnosis and decision support (CDSS) module for connecting with real-time vital signs and clinical indicators to generate diagnostic suggestions and treatment pathways to assist doctors in decision-making; an intelligent early warning and intervention module for responding to sudden changes in the patient's condition in real time, pushing intervention suggestions, and improving rescue efficiency; a discharge management and follow-up module for providing long-term care and follow-up support for high-risk newborns such as premature infants, and generating personalized health management plans; and a primary care adaptation and remote collaboration module for supporting lightweight deployment and serving remote collaborative consultations, training, and guidance in primary care NICUs.

[0095] The system in this embodiment operates based on a closed-loop logic of "screening-diagnosis-treatment-management". Specifically, screening involves collecting data at birth to intelligently identify high-risk newborns; diagnosis involves accessing real-time monitoring data from the NICU and medical records, with the model performing rapid diagnosis and disease correlation analysis; treatment involves generating personalized intervention pathways based on model suggestions to assist doctors in treatment decisions; and management involves covering discharge guidance, home care, regular follow-ups, and remote monitoring to achieve full life-cycle health management.

[0096] In this embodiment, a neonatal medical semantic network integrating knowledge graphs is constructed; a retrieval-enhanced generation (RAG) mechanism is introduced to improve knowledge accuracy and answer reliability; unified representation of multimodal data and heterogeneous data linkage reasoning are supported; and open interfaces are provided to support deployment in different scenarios, including centralized cloud services and local edge deployments.

[0097] The system application scenarios in this embodiment include: neonatal intensive care units (NICUs) in hospitals at all levels, newborn birth defect / genetic metabolic disease screening centers, premature infant tracking and management in community hospitals and maternal and child health institutions, medical teaching and intelligent question-and-answer assistance systems, and regional information supervision platforms of health administrative departments. For example, in a hospital NICU scenario, the system intelligently identifies early signs of sepsis in premature infants by accessing real-time monitoring data such as heart rate, blood oxygen, and respiration, combined with symptom descriptions and laboratory test indicators in electronic medical records. The system provides early warnings and pushes intervention plans before doctors intervene, achieving early intervention and quality improvement in traditional rescue procedures.

[0098] This invention constructs a professional knowledge-enhanced AI model to achieve intelligent services across the entire process of newborn screening, diagnosis, treatment, and management, thereby improving diagnostic efficiency and quality, reducing misdiagnosis and missed diagnosis rates, and promoting the standardization and homogenization of medical treatment. Specifically, it has the following beneficial effects: Rapid and accurate diagnosis and treatment response: Responds to changes in patient condition within seconds, improving the success rate of NICU critical care resuscitation; Professional knowledge and accurate context: Embedded with neonatal specialist knowledge bases and guidelines, avoiding AI illusions; Comprehensive coverage of the entire diagnosis and treatment process: Supports full-cycle management from birth screening to subsequent follow-up; Strong data fusion and processing capabilities: Supports simultaneous processing of images, text, tables, and monitoring equipment data; Scalable and replicable: Adaptable to deployment in primary-level NICUs, empowering the homogenization of regional medical services; Supports talent training: The system automatically generates explanatory paths and decision-making basis, facilitating teaching and training.

[0099] like Figure 2 As shown in the figure, an embodiment of the present invention provides a method for neonatal intensive care management based on an AI large model, comprising the following steps: S1. Acquire multimodal medical data and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; S2. Based on the dataset, the preset large language model is trained to obtain the neonatal critical care health management model. During the training process, the nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints and input into the preset large language model. And / or, the target information is retrieved from the knowledge graph using the retrieval enhancement generation mechanism, and then combined with the preset large language model to generate a personalized health management plan. S3. Based on the vital sign data of the target newborn and using the neonatal critical care health management model, generate a corresponding personalized health management plan.

[0100] Optionally, in the above technical solution, a retrieval-enhanced generation mechanism is used to retrieve target information from a knowledge graph, and then combined with a preset large language model to generate a personalized health management plan, including: Based on data from the dataset, target information is retrieved from the knowledge graph, including clinical guidelines and pathological features; The target information is aligned and corrected with the inference output of the preset large language model to generate a personalized health management plan.

[0101] Optionally, the above technical solution also includes: S4. Deploy the neonatal critical care health management model to different terminals, or make the neonatal critical care health management model lightweight and then deploy the lightweight neonatal critical care health management model to different terminals to enable real-time response to remote requests and push of personalized health management plans.

[0102] Optionally, in the above technical solution, the multimodal medical data includes: real-time vital sign data from monitoring equipment, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

[0103] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0104] It should be noted that the beneficial effects of the AI-based large-scale model-based neonatal intensive care management method provided in the above embodiments are the same as those of the AI-based large-scale model-based neonatal intensive care management system, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0105] The AI-based large-scale neonatal critical care management system of the present invention can be a computer program (including program code) running on a computer device. For example, the AI-based large-scale neonatal critical care management system of the present invention is an application software that can be used to execute the corresponding steps in the AI-based large-scale neonatal critical care management method of the present invention.

[0106] In some embodiments, the AI-based large-scale neonatal critical care management system of the present invention can be implemented in a combination of hardware and software. As an example, the AI-based large-scale neonatal critical care management system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the AI-based large-scale neonatal critical care management method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0107] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0108] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for neonatal critical care management based on an AI large model. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the method for neonatal critical care management based on an AI large model shown in any embodiment of the present invention by calling the computer program.

[0109] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0110] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for neonatal critical care management based on an AI large model.

[0111] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A neonatal critical care management system based on an AI large-scale model, characterized in that, include: The module includes a data access and integration module, a model training module, and a personalized health suggestion generation module. The data access and integration module is used to: acquire multimodal medical data, and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; The model training module is used to: train a preset large language model based on a dataset to obtain a neonatal critical care health management model. During the training process, nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints and input into the preset large language model. And / or, using a retrieval enhancement generation mechanism, target information is retrieved from the knowledge graph and then combined with the preset large language model to generate a personalized health management plan. The personalized health recommendation generation module is used to generate a corresponding personalized health management plan based on the vital sign data of the target newborn and using the neonatal critical care health management model.

2. The neonatal critical care management system based on an AI large model according to claim 1, characterized in that, Using a retrieval-enhanced generation mechanism, target information is retrieved from the knowledge graph, and then combined with a preset large language model to generate a personalized health management plan, including: Based on data from the dataset, target information is retrieved from the knowledge graph, including clinical guidelines and pathological features; The target information is aligned and corrected with the inference output of the preset large language model to generate a personalized health management plan.

3. The neonatal critical care management system based on an AI large model according to claim 1, characterized in that, It also includes a remote collaboration module, which is used for: The neonatal critical care health management model can be deployed to different terminals, or the neonatal critical care health management model can be lightweighted and then deployed to different terminals to enable real-time response to remote requests and push of personalized health management plans.

4. A neonatal intensive care management system based on an AI large model according to any one of claims 1 to 3, characterized in that, The multimodal medical data includes: real-time vital signs data from monitoring devices, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

5. A method for neonatal intensive care management based on an AI large-scale model, characterized in that, include: Acquire multimodal medical data and perform cross-source heterogeneous fusion processing on the multimodal medical data to generate a dataset; Based on the dataset, a pre-set large language model is trained to obtain a neonatal critical care health management model. During the training process, nodes and relationships in the knowledge graph constructed based on the neonatal specialist knowledge base are used as constraints and input into the pre-set large language model. And / or, a retrieval enhancement generation mechanism is used to retrieve target information from the knowledge graph, and then combined with the pre-set large language model to generate a personalized health management plan. Based on the vital signs data of the target newborns and using the neonatal critical care health management model, a corresponding personalized health management plan is generated.

6. The method for neonatal intensive care management based on an AI large model according to claim 5, characterized in that, Using a retrieval-enhanced generation mechanism, target information is retrieved from the knowledge graph, and then combined with a preset large language model to generate a personalized health management plan, including: Based on data from the dataset, target information is retrieved from the knowledge graph, including clinical guidelines and pathological features; The target information is aligned and corrected with the inference output of the preset large language model to generate a personalized health management plan.

7. The method for neonatal intensive care management based on an AI large model according to claim 5, characterized in that, Also includes: The neonatal critical care health management model can be deployed to different terminals, or the neonatal critical care health management model can be lightweighted and then deployed to different terminals to enable real-time response to remote requests and push of personalized health management plans.

8. A method for neonatal intensive care management based on an AI large model according to any one of claims 5 to 7, characterized in that, The multimodal medical data includes: real-time vital signs data from monitoring devices, test indicators from laboratory information systems, medical images from imaging systems, and text medical records from electronic medical record systems.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for neonatal critical care management based on an AI large model as described in any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the neonatal critical care management method based on an AI large model as described in any one of claims 5 to 8.

Images