Intelligent nursing record generation method and device

Abstract

The invention discloses an intelligent nursing record generation method and device, and the method comprises the steps: obtaining nursing voice data, and carrying out the noise reduction and sound enhancement processing to obtain a preprocessed voice stream; a speech recognition technology fusing ECAPA-TDNN and an x vector is adopted to recognize medical terminologies, and a text transcription result is generated; recognizing the voice of the target nurse from the preprocessed voice stream and the transcription result in combination with a speaker-independent model and a speaker condition model to obtain a voice transcription text of the voice; performing voice analysis and structured processing on the text by using a multi-modal self-supervised learning model and a hierarchical CNN-BiLSTM framework to obtain a structured text with clinical semantic features; key clinical information is extracted in combination with the medical ontology knowledge base, and a nursing record element set is obtained; and based on the nursing record element set, automatically generating a nursing record meeting the specification. According to the invention, the automatic generation from the nursing voice to the standardized nursing record is realized, and the efficiency and accuracy of the medical nursing record are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and in particular to an intelligent nursing record generation method and device for automatically converting nursing information into standardized electronic nursing records. Background Art

[0002] Medical care informatization is a crucial component of the modern healthcare system. Nursing records, as the core of patient medical data, are crucial for medical decision-making, nursing quality, and patient safety. The accuracy, timeliness, and standardization of nursing records directly impact the overall effectiveness of clinical care and the operational efficiency of healthcare institutions.

[0003] Traditional nursing record-keeping methods rely primarily on manual recording by nurses or keystrokes into electronic medical record systems. For example, common nursing record-keeping systems require nurses to enter relevant information on a computer keyboard after completing a nursing procedure before returning to their workstation. While some advanced hospitals have introduced mobile nursing terminals, allowing nurses to record information instantly at the bedside, manual touchscreen input is still required.

[0004] Currently, more advanced nursing record-keeping technologies are beginning to incorporate speech recognition into the nursing workflow, reducing the workload of manual data entry by converting nurses' spoken words into text. These systems typically use general-purpose speech recognition engines for speech-to-text conversion, then perform simple text processing to populate the content into pre-set templates. However, this approach suffers from low recognition accuracy when dealing with medical terminology, unstructured narrative content, and complex nursing scenarios, and struggles to automatically extract key clinical information and form a standardized structure.

[0005] The main problems with existing technologies include: inaccurate recognition of medical terminology, which requires nurses to frequently perform manual proofreading and revisions; an inability to effectively handle the challenges of speech recognition in noisy clinical environments; a lack of the ability to intelligently convert unstructured oral content into a standardized nursing record format; and a failure to automatically extract and highlight key clinical information, increasing the risk of important medical information being overlooked. Summary of the Invention

[0006] The purpose of the present invention is to provide an intelligent nursing record generation method and device, aiming to solve technical problems in the prior art such as inaccurate recognition of medical professional terms, difficulty in speech recognition in noisy environments, difficulty in converting unstructured oral content into standardized records, and difficulty in automatically extracting key clinical information.

[0007] To achieve the above objectives, the present invention provides an intelligent nursing record generation method, comprising:

[0008] Acquiring nursing voice data, performing noise reduction and sound enhancement processing on the nursing voice data, and obtaining a pre-processed nursing voice data stream;

[0009] Based on the pre-processed nursing voice data stream, medical professional terminology is recognized using speech recognition technology that integrates ECAPA-TDNN and x-vector to obtain text transcription results;

[0010] Recognizing the speech of the target nursing staff by combining a speaker-independent model and a speaker-conditional model with the preprocessed nursing speech data stream and the text transcription result to obtain a target speaker speech transcription text;

[0011] Based on the target speaker's speech transcription text, a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework are used to perform speech analysis and structural processing to obtain structured text with clinical semantic features;

[0012] Based on the structured text with clinical semantic features, combined with the medical ontology knowledge base, a deep learning key information extraction algorithm is used to extract key clinical information from the nursing record to obtain a standardized nursing record element set;

[0013] Based on the standardized nursing record element set, a nursing record document that complies with nursing standards is automatically generated through a template generation algorithm to obtain a standardized nursing record.

[0014] Preferably, the performing noise reduction and sound enhancement processing on the nursing voice data to obtain a pre-processed nursing voice data stream includes:

[0015] Using an adaptive noise cancellation algorithm to perform real-time identification and filtering of medical scenario-specific noise in the nursing voice data to obtain preliminary noise-reduced voice data;

[0016] Based on the preliminarily denoised speech data, spectral subtraction and multi-microphone beamforming technology are applied to enhance the human voice frequency band signal and suppress environmental reverberation, thereby obtaining acoustically enhanced high-quality speech data;

[0017] The acoustically enhanced high-quality voice data is segmented into breathing blocks using Silero voice activity detection technology and timestamps and scene identifiers are added to obtain the pre-processed nursing voice data stream.

[0018] Preferably, the speech recognition technology integrating ECAPA-TDNN and x-vector is used to identify medical terminology and obtain text transcription results, including:

[0019] Based on the preprocessed nursing voice data stream, an ECAPA-TDNN network is used to extract voice features and a medical terminology-optimized speech recognition model is constructed to obtain a preliminary voice feature representation;

[0020] receiving the preliminary speech feature representation, integrating ECAPA-TDNN and x-vector speaker representation technology to establish a decoupled representation of speaker identity and speech content through a contrastive learning method, thereby obtaining an enhanced speech feature with speaker information;

[0021] Based on the enhanced speech features with speaker information, an attention mechanism Transformer model optimized for a medical terminology library is applied to perform speech-to-text conversion to obtain the text transcription result.

[0022] Preferably, the identifying the speech of the target caregiver by combining the technology of the speaker-independent model and the speaker-conditional model to obtain the target speaker speech transcription text includes:

[0023] Using a speaker-independent model, a binary classifier is trained on the preprocessed nursing speech data stream and the text transcription result to automatically detect multi-speaker scenes in the speech and obtain time markers of speech overlapping areas;

[0024] The x-vector technology is used to extract the voiceprint features of each speaker and match them with the pre-registered nursing staff voiceprint database to obtain the speaker identification result;

[0025] Based on the speaker identification result and the time mark of the speech overlapping area, the feature linear modulation technology is used to integrate the speaker embedding and the speaker conditional model is applied to process the overlapping speech part to obtain the target speaker speech transcription text.

[0026] Preferably, the multimodal self-supervised learning model and the hierarchical CNN-BiLSTM framework are used to perform speech analysis and structural processing to obtain structured text with clinical semantic features, including:

[0027] Using pre-trained Wav2Vec2, HuBERT, and WavLM self-supervised learning models to extract acoustic and semantic features of the target speaker's speech transcription text from different dimensions to obtain a multimodal feature representation;

[0028] Dividing the multimodal feature representation into breathing chunks based on Silero voice activity detection technology and analyzing chunk-level fluency metrics including speech rate, pause duration, and repetition pattern to obtain prosodic speech features;

[0029] The outputs of different self-supervised learning models are fused through a learnable weighting mechanism, combined with the prosodic speech features, to balance the contributions of acoustic and linguistic features to obtain a comprehensive feature representation;

[0030] A hierarchical CNN-BiLSTM network architecture is applied to capture local and long-term dependencies and perform deep semantic analysis on the comprehensive feature representation to obtain the structured text with clinical semantic features.

[0031] Preferably, the medical ontology knowledge base is combined with a deep learning key information extraction algorithm to extract key clinical information from nursing records to obtain a standardized nursing record element set, including:

[0032] Based on clinical nursing standards and professional guidelines, a multi-dimensional medical ontology knowledge base including nursing operations, drugs, symptoms, and signs is constructed to obtain a structured medical ontology knowledge model;

[0033] Using the structured medical ontology knowledge model, performing entity recognition and relationship extraction on the structured text with clinical semantic features, mapping the descriptive language in the structured text with clinical semantic features to standard medical terminology to obtain a standardized clinical terminology set;

[0034] Applying a deep learning key information extraction algorithm to automatically identify and rate important clinical information in the standardized clinical terminology set to obtain a clinical information set with importance tags;

[0035] Classifying and arranging the clinical information set with importance marks into element groups that conform to the nursing record structure according to the element model of the nursing record standard to obtain the standardized nursing record element set;

[0036] The medical ontology knowledge model is continuously learned and updated, and the structure of the medical ontology knowledge base is optimized according to newly added clinical cases and nursing records to obtain a dynamically updated medical ontology knowledge model.

[0037] Preferably, the template generation algorithm automatically generates a nursing record document that complies with nursing standards to obtain a standardized nursing record, including:

[0038] Based on the hospital's existing nursing record specifications and electronic medical record standards, a template library covering different nursing scenarios was constructed to obtain a dynamic nursing record template set;

[0039] According to the content characteristics and context information of the standardized nursing record element set, a template matching algorithm is applied to automatically select the most suitable nursing record template to obtain a matching record template;

[0040] Intelligently filling the clinical information in the standardized nursing record element set according to the matched record template and adding metadata including a timestamp and an operation type to obtain a preliminarily formatted nursing record;

[0041] Using medical common sense verification rules and completeness check algorithms to identify missing items and logical errors in the initially formatted nursing records, and obtain intelligent completion suggestions;

[0042] After confirmation by the nursing staff, the initially formatted nursing record is optimized according to the intelligent completion suggestion to obtain the standardized nursing record.

[0043] Preferably, the application of a deep learning key information extraction algorithm to automatically identify and rate important clinical information in the standardized clinical terminology set to obtain a clinical information set with importance tags includes:

[0044] Constructing a bidirectional long short-term memory network model based on an attention mechanism to semantically encode clinical terms in the standardized clinical term set to obtain a semantic vector representation of the term;

[0045] Using a conditional random field model to perform sequence annotation on the semantic vector representation of the term to identify key clinical entity types and obtain entity annotation results;

[0046] Applying a graph convolutional neural network to analyze the dependencies and semantic associations between entities in the entity annotation results to obtain an entity relationship graph;

[0047] An importance scoring model is constructed based on medical expert knowledge and clinical guidelines to quantitatively evaluate the importance of the clinical information in the entity relationship diagram to obtain an importance score;

[0048] Priority marks and urgency marks are added to the clinical information in the entity relationship diagram according to the importance score to obtain the clinical information set with importance marks.

[0049] Preferably, the method of integrating speaker embedding with a feature linear modulation technique and applying the speaker conditional model to process overlapping speech parts to obtain the target speaker speech transcript includes:

[0050] Inputting the speaker embedding vector in the speaker identification result into a feature linear modulation network, dynamically modulating the speech features by learning scaling and offset parameters to obtain speaker conditional features;

[0051] Constructing a speaker-conditional model and training it on the synthesized mixed speech data so that the speaker-conditional model focuses on the speech content of the target speaker, thereby obtaining a trained speaker-conditional model;

[0052] Using the trained speaker conditional model, the overlapping speech segments corresponding to the time markers of the speech overlapping regions are processed to obtain separated speech content of the target nursing staff;

[0053] Establish a dynamic speaker tracking mechanism to associate the voice clips of the same nurse in different time periods through continuous matching of voiceprint features to obtain the associated voice clips;

[0054] The associated speech segments are integrated into a complete narrative based on time sequence and semantic coherence while retaining timestamp information to obtain a speech transcription text of the target speaker.

[0055] The present invention also provides an intelligent nursing record generating device, comprising:

[0056] A voice data preprocessing module is used to obtain nursing voice data, perform noise reduction and sound enhancement processing on the nursing voice data, and obtain a preprocessed nursing voice data stream;

[0057] A speech recognition module is used to identify medical terminology based on the preprocessed nursing speech data stream using speech recognition technology that integrates ECAPA-TDNN and x-vector to obtain text transcription results;

[0058] a target speaker recognition module for identifying the speech of a target caregiver by combining a speaker-independent model and a speaker-conditional model with the preprocessed nursing speech data stream and the text transcription result to obtain a target speaker speech transcription text;

[0059] A speech analysis and structured processing module is used to transcribe the target speaker's speech into text, perform speech analysis and structured processing using a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework, and obtain structured text with clinical semantic features;

[0060] A clinical information extraction module is used to extract key clinical information from nursing records based on the structured text with clinical semantic features and in combination with a medical ontology knowledge base through a deep learning key information extraction algorithm to obtain a standardized set of nursing record elements;

[0061] The nursing record generation module is used to automatically generate a nursing record document that complies with nursing standards based on the standardized nursing record element set through a template generation algorithm to obtain a standardized nursing record.

[0062] The beneficial effects of the present invention are:

[0063] 1. By combining a speaker-independent model with a speaker-conditional model, the system achieves accurate separation and recognition of the target nurse's speech in noisy medical environments, solving the speech recognition problem in multi-speaker scenarios.

[0064] 2. The use of speech recognition technology that integrates ECAPA-TDNN and x-vector significantly improves the recognition accuracy of medical terminology and reduces the workload of manual proofreading and modification;

[0065] 3. By combining a multimodal self-supervised learning model with a hierarchical CNN-BiLSTM framework, we achieve deep semantic understanding and structured processing of nursing speech content, converting unstructured oral content into standardized nursing records.

[0066] 4. Based on the medical ontology knowledge base and deep learning key information extraction algorithm, it can automatically identify and mark important clinical information, improving the accuracy and completeness of nursing records;

[0067] 5. Through dynamic nursing record template generation and intelligent filling mechanism, the automatic generation of nursing records and seamless integration with the electronic medical record system are achieved, greatly improving the efficiency of nursing records. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 A flowchart of a method for generating intelligent nursing records according to an embodiment of the present invention;

[0070] Figure 2 A flowchart of medical professional speech recognition and transcription provided by an embodiment of the present invention;

[0071] Figure 3 This is a structural block diagram of the intelligent nursing record generation device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, only portions relevant to the present invention are shown in the accompanying drawings.

[0073] Please refer to Figure 1 , Figure 1Flowchart of the method for generating intelligent nursing records provided by an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0074] Step S101: Acquire nursing voice data, perform noise reduction and sound enhancement processing on the nursing voice data, and obtain a pre-processed nursing voice data stream.

[0075] In this step, intelligent recording devices first capture the nursing information spoken by nurses in medical settings, generating raw nursing voice data. These intelligent recording devices can be portable microphones worn by nurses, directional sound pickup devices in wards, or voice capture devices integrated into mobile nursing terminals. Because medical environments are often subject to various noise disturbances, such as monitor alarms, infusion pump sounds, and conversations between patients and visitors, the collected raw voice data requires noise reduction and sound enhancement processing. The noise reduction process utilizes an algorithm optimized for the specific noise patterns in medical settings, effectively identifying and filtering these specific noises. Sound enhancement processing focuses on improving signal quality in the human voice frequency band, enhancing the clarity of the nurses' spoken content. Furthermore, the processed voice data is segmented and labeled, adding metadata such as timestamps and scene identifiers to facilitate subsequent processing and traceability. This series of processes produces a high-quality, pre-processed nursing voice data stream, laying the foundation for subsequent speech recognition and analysis.

[0076] Step S102: Based on the pre-processed nursing voice data stream, medical professional terminology recognition is performed using a speech recognition technology that integrates ECAPA-TDNN and x-vector to obtain a text transcription result.

[0077] In this step, a speech recognition technology combining ECAPA-TDNN and x-vector was employed to address the challenge of recognizing medical terminology. ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation-Time Delay Neural Network) is an improved time-delay neural network specifically suited for processing temporal features and medical terminology in speech signals. First, the ECAPA-TDNN network was used to extract speech features from the preprocessed speech data stream, building a basic speech recognition model optimized for medical terminology. Then, by combining ECAPA-TDNN with x-vector speaker representation technology, a contrastive learning approach was used to establish a decoupled representation of speaker identity and speech content. This decoupled representation enables better differentiation of speech content between different speakers, improving recognition accuracy in multi-person conversation environments. Next, a Transformer model with an attention mechanism optimized for a medical terminology corpus was applied to perform speech-to-text conversion. This model, specifically trained on medical corpora, accurately recognizes various medical terminology, drug names, dosage expressions, and other specialized content. Ultimately, accurate text transcription results are output, retaining the medical professional terms and key information in the original speech, providing a text basis for subsequent target speaker recognition and content analysis.

[0078] Step S103: The pre-processed nursing voice data stream and the text transcription result are used to identify the target nursing staff's voice by combining a speaker model-independent and speaker conditional model-based technology to obtain a target speaker's voice transcription text.

[0079] This step addresses common multi-speaker scenarios in healthcare settings, such as nurse-patient conversations and multi-staff discussions. First, a binary classifier is trained on the preprocessed nursing speech data stream and text transcriptions using a speaker-independent (SI) model. This automatically detects the presence of multiple speakers in the speech and marks the time periods where the speech overlaps. Next, x-vector technology is used to extract the voiceprint features of each speaker and match them against a pre-established database of nursing staff voiceprints to identify the speaker corresponding to each portion of the speech. Speakers not registered in the voiceprint database (such as patients or visitors) are marked as "unregistered speakers" and assigned a temporary ID. Next, based on the speaker identification results and the time stamps of the overlapping speech regions, feature linear modulation (FiLM) is used to integrate speaker embeddings, and a speaker-conditional (SC) model is applied to process the overlapping speech portions. For non-overlapping regions, the speaker-independent model is used to process the speech content directly, retaining the portions identified as belonging to the target caregiver. A dynamic speaker tracking mechanism has also been established. By continuously matching voiceprint features, it links voice clips belonging to the same caregiver from different time periods to form a coherent transcript of the target speaker's speech. This process accurately extracts the target caregiver's speech content in complex multi-person conversation scenarios, providing clean text input for subsequent semantic analysis.

[0080] Step S104: Based on the target speaker's speech transcription text, a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework are used to perform speech analysis and structural processing to obtain a structured text with clinical semantic features.

[0081] In this step, the target speaker's speech transcript undergoes deep semantic analysis and structural processing. First, three pre-trained self-supervised learning models—Wav2Vec2, HuBERT, and WavLM—are used to extract acoustic and semantic features from different dimensions. Each model has its own strengths: Wav2Vec2 excels at extracting basic acoustic features, HuBERT excels at identifying speech units and phonemes, and WavLM is more suitable for handling complex acoustic environments. The speech signal is input into these three models, extracting features from different layers, including low-level acoustic information, mid-level prosodic information, and high-level semantic information. Then, based on Silero voice activity detection technology, the multimodal feature representation is partitioned into breathing chunks. Chunk-level fluency metrics such as speaking rate, pause duration, stress patterns, n-gram repetitions, and pitch variation are analyzed to generate prosodic speech features. Next, the outputs of the different self-supervised learning models are fused using a learnable weighting mechanism. The weight allocation strategy is dynamically adjusted based on the different types of nursing content to balance the contributions of acoustic and linguistic features, resulting in a comprehensive feature representation. Finally, a hierarchical CNN-BiLSTM network architecture is applied to perform deep semantic analysis on the integrated feature representation. The CNN layer captures local features and patterns, while the BiLSTM layer captures long-term dependencies and contextual information. The network's hierarchical design enables simultaneous processing of language units at varying granularities, from word to paragraph level. Each layer performs entity recognition, relationship extraction, temporal reasoning, and logical inference, ultimately outputting structured text with clinical semantic features, including elements such as the nursing procedure type, procedure object, numerical parameters, time information, observations, and importance tags.

[0082] Step S105: Based on the structured text with clinical semantic features, combined with the medical ontology knowledge base, the key clinical information in the nursing record is extracted through a deep learning key information extraction algorithm to obtain a standardized nursing record element set.

[0083] In this step, a multi-dimensional medical care ontology knowledge base, encompassing nursing procedures, medications, symptoms, and signs, is constructed based on clinical nursing standards and professional guidelines. This knowledge base, comprising components such as a concept hierarchy, entity attribute definitions, a relationship network, and rule constraints, provides a foundation for extracting and standardizing clinical information. This structured medical ontology knowledge model is used to perform entity recognition and relationship extraction on structured text with clinical semantic features, mapping descriptive language within the text to standard medical terminology. This process, which includes medical entity recognition, term standardization, synonym processing, abbreviation expansion, and numerical unit standardization, transforms non-standard and diverse descriptive language into standardized, unified medical terminology, forming a standardized clinical terminology set. Next, a deep learning-based key information extraction algorithm is applied to automatically identify and rank important clinical information within the text. A bidirectional long short-term memory network model based on an attention mechanism is constructed to semantically encode clinical terms. Sequence annotation is performed using a conditional random field model. A graph convolutional neural network is applied to analyze dependencies and semantic associations between entities. Finally, an importance scoring model, based on medical expert knowledge and clinical guidelines, is constructed to assign priority and urgency tags to clinical information. Next, based on the standardized element model of nursing records (such as SOAP format or a hospital-defined format), clinical information with importance marks is categorized and organized into element groups that conform to the nursing record structure, forming a standardized nursing record element set. Furthermore, a continuous learning and updating mechanism for the medical ontology knowledge model is established. Through the collection of new terms, relationship discovery, concept hierarchy adjustment, rule optimization, and knowledge graph expansion, the knowledge base can be continuously optimized and expanded as new clinical cases and nursing records are added.

[0084] Step S106: Based on the standardized nursing record element set, a nursing record document that complies with nursing standards is automatically generated through a template generation algorithm to obtain a standardized nursing record.

[0085] In this step, based on the hospital's existing nursing record specifications and electronic medical record standards, a template library covering various nursing scenarios was constructed, including templates for basic nursing records, specialized nursing records, special nursing procedures, assessment records, and shift handover records. Each template defines a specific structure and fields, including required, optional, and automatically calculated fields. A template matching algorithm was applied to automatically select the most suitable nursing record template based on the content characteristics and contextual information of the standardized nursing record element set. The template matching algorithm made decisions based on factors such as content type matching, keyword matching, completeness assessment, contextual analysis, and historical references. A matching score was calculated for each candidate template, and the template with the highest score was selected as the final matching result. Clinical information from the standardized nursing record element set was then intelligently populated according to the matching record template, and metadata such as timestamp, operation type, record status, and identifiers were added. The intelligent population process includes field mapping, automatic calculation, format conversion, default value filling, and cross-validation. Subsequently, a quality check of the initially formatted nursing record was performed using medical common sense validation rules and a completeness check algorithm to identify potential missing items and logical errors. Verifications are performed, including completeness checks, logical consistency checks, medical plausibility checks, time series checks, and numerical range checks. Intelligent completion suggestions are generated, including missing item prompts, outlier warnings, logical error prompts, and reference value suggestions. Finally, the intelligent completion suggestions are presented to the nursing staff, and upon confirmation, the nursing records are optimized to generate standardized nursing records. These records are not only accurate and complete in content, but also standardized and formatted to meet the hospital's electronic medical record requirements. A record review function is also provided, allowing the head nurse or physician to review and make necessary corrections to the automatically generated records. Feedback from nursing staff is also collected for continuous improvement and optimization of the record generation algorithm.

[0086] The above steps are described in detail below.

[0087] Step S101 includes the following sub-steps:

[0088] Step S1011: using an adaptive noise elimination algorithm to perform real-time identification and filtering of medical scenario-specific noise in the nursing voice data to obtain preliminarily noise-reduced voice data.

[0089] Common noises in medical settings include monitor alarms, infusion pumps, medical equipment operation, and conversations between patients and visitors. These noises can significantly impact speech recognition accuracy. In this step, an adaptive noise cancellation algorithm is used to address these specific noises in medical settings. The algorithm first establishes a medical noise signature library containing the characteristic patterns of various common medical devices. Upon receiving raw nursing voice data, the algorithm analyzes the input signal's spectral characteristics in real time, identifies the noise components, and matches them against the pre-defined noise signature library. For the identified specific noises, the algorithm adaptively adjusts filter parameters, employing different filtering strategies for different noise types. For example, a notch filter is used for continuous equipment operation sounds, while time-domain suppression techniques are employed for sudden alarms. Furthermore, the algorithm dynamically adjusts parameters based on changes in the noise environment, adapting to the noise characteristics of different wards and time periods. This process effectively filters out specific noises in medical settings while preserving the original characteristics of the nurse's voice to the greatest extent possible. The resulting output is preliminarily noise-reduced voice data, laying the foundation for subsequent voice enhancement processing.

[0090] Step S1012: Based on the preliminarily denoised speech data, spectrum subtraction and multi-microphone beamforming technology are applied to enhance the human voice frequency band signal and suppress environmental reverberation, thereby obtaining acoustically enhanced high-quality speech data.

[0091] In this step, the preliminarily noise-reduced speech data undergoes further sound enhancement processing, focusing on improving signal quality in the human voice frequency band. First, spectral subtraction technology is applied to estimate the noise power spectrum and then subtract it from the speech signal's power spectrum. This process specifically focuses on the primary frequency band of human voice (300Hz-3400Hz), effectively improving the signal-to-noise ratio in this frequency band. An improved spectral subtraction algorithm uses adaptive smoothing to reduce the "musical noise" that can be introduced by traditional spectral subtraction, thereby maintaining the naturalness of the speech. Furthermore, for acquisition devices equipped with multi-microphone arrays, beamforming technology is applied to further enhance the speech signal in the target direction (typically the nurse's position) while suppressing noise and interference from other directions. Beamforming adjusts the phase and amplitude of each microphone signal to create a directional reception pattern, achieving spatial filtering. Several commonly used microphone array configurations are pre-configured to suit different care environments, tailored to the specific needs of medical settings. Furthermore, dereverberation processing is applied to reduce the impact of sound reflections in the indoor environment on speech clarity. By combining spectral subtraction and beamforming technology, the signal-to-noise ratio of speech signals can be significantly improved, and high-quality speech data after acoustic enhancement can be obtained, providing good input for subsequent speech activity detection and segmentation processing.

[0092] Step S1013: using Silero voice activity detection technology to segment the acoustically enhanced high-quality voice data into breathing blocks and add timestamps and scene identifiers to obtain the pre-processed nursing voice data stream.

[0093] In this step, Silero Voice Activity Detection (Silero-VAD) technology is used to finely segment the acoustically enhanced, high-quality speech data. Silero-VAD is an efficient voice activity detection tool that can accurately identify voiced and silent segments in speech, making it particularly suitable for processing speech data in medical scenarios. The continuous speech stream is segmented according to the speaker's natural breathing pauses. This segmentation method is more consistent with the natural rhythm of human speech than fixed time windows and can better preserve semantic integrity. For each identified speech segment, its acoustic features are analyzed to determine whether it is a complete semantic unit. If the speech segment is detected to be too long, it is further segmented at appropriate pauses to ensure that each segment is a suitable length for processing. At the same time, accurate timestamp information is added to each segment to record the start and end time of the speech, facilitating subsequent processing and time tracing in medical records. In addition, scene identification information such as ward number, bed number, and nursing operation type is also added. This information can be obtained from nursing schedules, hospital information, or manual input by nurses. Through this processing, continuous speech data is converted into a collection of speech segments with structured metadata, forming a preprocessed nursing speech data stream, which provides a good foundation for subsequent speech recognition and multi-speaker scenario processing.

[0094] Please refer to Figure 2 , Figure 2 This is a flowchart of medical professional speech recognition and transcription provided by an embodiment of the present invention. Figure 2 As shown, step S102 includes the following sub-steps:

[0095] Step S1021: Based on the preprocessed nursing voice data stream, the ECAPA-TDNN network is used to extract voice features and construct a medical terminology optimized speech recognition model to obtain a preliminary voice feature representation.

[0096] In this step, the preprocessed nursing speech data stream output from step S1013 is received. This speech data has been subjected to noise reduction and sound enhancement, segmented into breathing chunks, and includes metadata tags such as timestamps and scene identifiers. Speech features are extracted using the ECAPA-TDNN (Emphasized Channel Attention, Propagation and Aggregation-Time Delay Neural Network) architecture. This improved time-delay neural network features enhanced channel attention, information propagation, and aggregation capabilities, making it particularly suitable for processing the specialized terminology found in medical speech. First, the speech data is converted into time-frequency domain feature representations such as spectrograms or Mel-frequency cepstral coefficients, which serve as input to the ECAPA-TDNN network. The network comprises multiple time-delay layers, capable of capturing speech features at different time scales and particularly well-suited for processing the long-term dependencies found in medical terminology. The network design incorporates an attention mechanism optimized for medical scenarios, enabling the model to automatically focus on portions of speech that may contain medical terminology, improving its sensitivity to key words. Furthermore, the model is specifically optimized for the characteristics of medical terminology, such as increasing the weight of medical vocabulary and adjusting acoustic model parameters. The model is pre-trained using a large amount of medical speech data, including various medical terminology, drug names, anatomical terms, diagnosis names, and nursing procedure terms, enabling high recognition accuracy for medical speech. This process converts the pre-processed nursing speech data stream into preliminary speech feature representations that retain key acoustic characteristics of the speech while enhancing the medical terminology, laying the foundation for zero-shot multi-speaker representation learning.

[0097] The ECAPA-TDNN network adopts a multi-layer architecture. The network consists of five TDNN layers, each with 512 channels and convolutional kernels of sizes 5, 3, 3, 1, and 1, respectively. Each TDNN layer is followed by batch normalization, ReLU activation, and dropout (with a ratio of 0.5). The network incorporates a Squeeze-and-Excitation (SE) module to enhance feature representation by learning inter-channel dependencies. The SE module uses a compression ratio of 8, effectively reducing the number of parameters while maintaining performance. The statistical pooling layer calculates the mean and standard deviation of frame-level features to generate utterance-level representations of fixed dimension (1024). The model is trained using a medical speech dataset containing approximately 5,000 hours of medical speech recordings covering over 10,000 medical terms. Training is performed using the Adam optimizer with an initial learning rate of 0.001, a cosine annealing strategy, a batch size of 64, and 200 epochs. To enhance the model’s ability to recognize medical terms, we oversampled samples of medical terms, increasing their proportion in the training data to 40%.

[0098] Step S1022: Receive the preliminary speech feature representation, integrate ECAPA-TDNN and x-vector speaker representation technology to establish a decoupled representation of speaker identity and speech content through a contrastive learning method, and obtain enhanced speech features with speaker information.

[0099] In this step, the preliminary speech feature representation from step S1021 is received. The goal is to build a robust model capable of handling speech from different speakers, even new speakers that have never been seen before. The ECAPA-TDNN and x-vector techniques are integrated to achieve a decoupled representation of speaker identity and speech content. The x-vector is an embedding representation used for speaker recognition that effectively captures speaker identity characteristics. First, the speech features extracted by the ECAPA-TDNN are fed into the x-vector extractor to obtain speaker identity features (speaker embeddings). Then, using contrastive learning, the model is trained to distinguish between speech content and identity features of different speakers. During contrastive learning, a triplet of samples (anchor, positive, negative) is constructed, where the anchor is a speech sample of the target speaker, the positive is another speech sample of the same speaker, and the negative is a speech sample of a different speaker. By minimizing the distance between the anchor and the positive while maximizing the distance between the anchor and the negative, the model learns to map different speech contents of the same speaker into similar identity spaces, while mapping the speech of different speakers into different identity spaces. Specially designed loss functions (such as triplet loss or contrastive loss) are used to enable the model to learn a decoupled representation of speech content from speaker identity. Furthermore, domain adaptation techniques are introduced to enable the model to adapt to specific speech characteristics in medical scenarios, such as the professional language habits of medical staff and speech changes when wearing masks. This process yields enhanced speech features with speaker information. These features incorporate both speech content and speaker identity information, but the two are decoupled in the feature space, providing a more robust foundation for subsequent medical term recognition.

[0100] The x-vector technology is implemented based on a modified time-delay neural network (TDNN) architecture. The network first processes frame-level features through five TDNN layers: the first layer processes the current frame t; the second layer processes frames t-2, t, and t+2; the third layer processes frames t-3, t, and t+3; the fourth layer processes frames t-4, t, and t+4; and the fifth layer processes all frames. Each layer contains 512 neurons and uses the Reluctant Unit (ReLU) activation function and batch normalization. A statistical pooling layer aggregates temporal information, calculates the mean and standard deviation, and generates a 1024-dimensional speaker embedding. Two subsequent fully connected layers (with sizes of 512 and 256, respectively) further compress the representation, ultimately outputting a fixed-dimensional x-vector. During contrastive learning training, a triplet loss function is used, defined as L = max(0, m + D(a,p) - D(a,n)), where m is a margin parameter (set to 0.2), D represents the cosine distance, a represents an anchor sample, p represents a positive sample (same speaker), and n represents a negative sample (different speakers). The model was trained on a dataset containing speech samples from 500 medical professionals, each providing approximately 20 minutes of recording. Data augmentation techniques, including adding background noise from medical environments, varying speech speed (0.9-1.1 times), and adjusting volume (±3dB), improved the model's adaptability to real-world scenarios.

[0101] Step S1023: Based on the enhanced speech features with speaker information, an attention mechanism Transformer model optimized for the medical terminology library is applied to perform speech-to-text conversion to obtain the text transcription result.

[0102] In this step, the core speech-to-text process is performed based on the enhanced speech features with speaker information output from step S1022, with particular attention paid to the accurate recognition of medical terminology. A Transformer model with an attention mechanism optimized for a medical terminology database is used for speech-to-text conversion. The Transformer model, with its powerful attention mechanism and parallel processing capabilities, is well suited for processing long sentences and specialized terminology found in medical speech. The enhanced speech features are input into a pre-trained Transformer encoder-decoder architecture. In the encoder, a multi-head self-attention mechanism is used to process the input speech feature sequence, capturing dependencies between different positions. In the decoder, the encoder output is combined with the generated partial text to generate the next word using self-attention and encoder-decoder attention mechanisms. The introduction of a medical-specific attention mechanism into the Transformer model enables the model to pay special attention to portions of speech that may contain medical terminology. For example, when a key word such as "blood pressure" or "infusion" is detected in speech, the model will pay more attention to the subsequent numerical value or description. It also integrates a dictionary of medical terminology, including standard drug names, anatomical terms, diagnosis names, nursing operation terms, etc., and introduces dictionary-based language model constraints in the decoding process to improve the recognition accuracy of professional terms. In addition, a language model trained with medical corpus is used to assist in decoding, improving the fluency of the generated text and the rationality of the medical context. The medical terms in the recognition results will be post-processed, including term standardization, synonym processing, abbreviation expansion and numerical unit standardization, to ensure that the output text meets the standard requirements of medical records. Through this processing, the speech is converted into accurate text transcription results containing medical professional terms, providing a basis for subsequent multi-speaker scenario processing.

[0103] Step S103 includes the following sub-steps:

[0104] Step S1031: using a speaker-independent model to train a binary classifier on the preprocessed nursing speech data stream and the text transcription result, automatically detecting multi-speaker scenes in the speech, and obtaining time markers of speech overlapping areas.

[0105] This step addresses common multi-speaker scenarios in medical settings, such as nurse-patient conversations and discussions among multiple medical staff. The preprocessed nursing speech data stream and the text transcription output from step S1023 are first received as input. The speaker-independent (SI) model is a general speech processing model that does not rely on specific speaker features and can extract essential speech features regardless of speaker identity. The SI model performs preliminary analysis of the speech, extracting basic acoustic features such as energy distribution, spectral characteristics, and fundamental frequency variation. Based on the SI model output, a lightweight binary classifier is trained to specifically detect areas of speaker overlap in the speech. This classifier utilizes a convolutional neural network architecture, taking as input the feature sequence extracted by the SI model and outputting a binary judgment indicating whether each time frame contains multiple speakers. The training process utilizes a large amount of annotated multi-speaker speech data, including conversations and discussions from various medical scenarios, enabling the classifier to accurately identify overlapping voices from different speakers. Furthermore, the text transcription results are combined to perform auxiliary judgments, such as detecting conversational sentence structure and question-answer patterns, to improve detection accuracy in multi-speaker scenarios. For each detected overlapping region, its start and end timestamps are recorded, forming a time stamp for the speech overlap. The innovation of this step lies in using the frozen SI model output as a feature basis for training a dedicated binary classifier, significantly reducing computational cost while maintaining high detection accuracy. While traditional methods typically require complete speaker separation to detect overlap, this method achieves efficient detection through simplified binary classification.

[0106] Step S1032: applying x-vector technology to extract the voiceprint features of each speaker and matching them with the pre-registered nursing staff voiceprint database to obtain the speaker identification result.

[0107] In this step, for the multi-speaker scenario detected in step S1031, x-vector technology is applied to extract the voiceprint features of each speaker. x-vectors are a powerful speaker representation method that can extract speaker identity features from short speech segments. The speech data is first framed, typically using a 25 millisecond frame length and a 10 millisecond frame shift. Acoustic features, such as Mel-Frequency Cepstral Coefficients (MFCCs) or Filter Bank Energy (FBANK) features, are then calculated for each frame. These frame-level features are then input into a deep neural network consisting of multiple time-delay layers to capture voiceprint features at different time scales. At the end of the network, a statistical pooling layer is used to convert the variable-length frame sequence into a fixed-dimensional speaker embedding representation, or x-vector. Simultaneously, a nursing staff voiceprint database is established, pre-collecting and registering the voiceprint features of hospital nursing staff. During the registration process, each nurse provides multiple speech samples, from which x-vectors are extracted and stored in the voiceprint database, along with their identity information (such as employee number and name). In actual use, algorithms such as cosine similarity or probabilistic linear discriminant analysis (PLDA) are used to calculate the similarity scores between the extracted x-vector and each template in the voiceprint library. Based on the similarity scores and preset thresholds, the speaker identity corresponding to each part of the speech is determined. Speakers not registered in the voiceprint library (such as patients or visitors) are marked as "unregistered speakers" and assigned a temporary ID for subsequent processing. The x-vector technology used achieves higher speaker recognition accuracy than traditional Gaussian mixture models (GMM) or i-vector methods, especially in short speech clips and noisy environments.

[0108] Step S1033: Based on the speaker identification result and the time stamp of the speech overlap area, the feature linear modulation technology is used to integrate the speaker embedding and the speaker conditional model is applied to process the overlapping speech part to obtain the target speaker speech transcription text.

[0109] In this step, the speech content of the target caregiver must be isolated in a multi-speaker scenario, specifically addressing areas of speech overlap. First, the speaker identification results and the time stamps of the speech overlap areas are received as input. For non-overlapping areas, the speech content is directly processed using a speaker-independent (SI) model, retaining the portions identified as belonging to the target caregiver. For overlapping areas, a speaker-conditioned (SC) model is applied. The SC model is a speech processing model optimized specifically for the target speaker, capable of extracting speaker-specific content from multi-speaker mixtures. The SC model incorporates speaker embeddings using the Feature Linear Modulation (FiLM) technique. FiLM is a conditional computation method that allows the model to dynamically adjust its processing parameters based on speaker embeddings. Specifically, the speaker embedding vector from the speaker identification results is input into a feature linear modulation network, which dynamically modulates the speech features by learning scaling and offset parameters to obtain speaker-conditioned features. A speaker-conditioned model is constructed and trained on synthesized mixed speech data, allowing the model to focus on the speech content of the target speaker. During the training process, a large amount of synthesized mixed speech data is generated, including the speech of the target speaker and various interfering speakers, enabling the model to learn the ability to separate the target speaker in a complex environment. After processing, the trained speaker conditioning model is used to process the overlapping speech segments corresponding to the time stamps of the overlapping speech areas to obtain the separated speech content of the target caregiver. In addition, a dynamic speaker tracking mechanism is established to continuously match voiceprint features to associate speech segments of the same caregiver from different time periods to obtain associated speech segments. Finally, based on chronological order and semantic coherence, the associated speech segments are integrated into a complete narrative while retaining timestamp information to obtain the target speaker's speech transcript. The innovation of this step lies in the combination of the SI model and the SC model, and the flexible integration of speaker conditioning through FiLM technology, which greatly improves the accuracy and efficiency of extracting the target caregiver's speech content in complex medical environments.

[0110] In a preferred embodiment, the method of integrating speaker embedding using feature linear modulation technology and applying the speaker conditional model to process overlapping speech parts to obtain the target speaker speech transcript includes:

[0111] The speaker embedding vector in the speaker identification result is input into a feature linear modulation network, and the speech features are dynamically modulated by learning scaling and offset parameters to obtain speaker conditional features.

[0112] Feature Linear Modulation (FiLM) is a powerful conditional computing technique that dynamically adjusts the feature representation of a neural network based on conditional information. In this embodiment, the speaker embedding vector of the target caregiver is input into the FiLM network as conditional information. The FiLM network consists of multiple fully connected layers that learn two sets of parameters: a scaling parameter γ and an offset parameter β. These parameters are used to modulate the speech feature map using the following formula: FiLM(h, z) = γ(z) × h + β(z), where h is the original speech feature map and z is the speaker embedding vector. In this way, the parameters of the speech processing network can be dynamically adjusted based on the target caregiver's voice characteristics, allowing the network to focus more on the target speaker's voice characteristics, enhancing its response to the target speaker's speech while suppressing interference from other speakers. This dynamic modulation mechanism enables adaptation to the voice characteristics of different caregivers, improving the flexibility and accuracy of speech separation.

[0113] A speaker-conditional model is constructed and trained on the synthesized mixed speech data so that the speaker-conditional model focuses on the speech content of the target speaker, thereby obtaining a trained speaker-conditional model.

[0114] The speaker-conditioned model is a speech processing model optimized for a specific speaker. It can extract the target speaker's content from multi-speaker mixtures. First, a large-scale synthetic mixed speech dataset is constructed, containing a mixture of the target speaker's speech and other speakers' speech in various scenarios. During the synthesis process, factors such as the mixing ratio, overlap, and background noise are controlled to simulate the complexities of real-world medical environments. During training, a supervised learning approach is used, with the target speaker's clean speech as the training target and the mixed speech as the input. The model parameters are optimized by minimizing the difference between the predicted output and the target. The model architecture uses a modified version of the Conv-TasNet (Time-Domain Audio Separation Network) combined with FiLM technology for conditioning. During training, a combination of multiple loss functions, including signal reconstruction loss, time-frequency masking loss, and perceptual quality loss, is used to comprehensively optimize model performance. Furthermore, data augmentation techniques, such as tempo perturbation, volume variation, and channel effect simulation, are employed to improve the model's generalization. Through this training process, a speaker-conditioned model is obtained that can effectively separate the target nurse's speech.

[0115] The trained speaker conditional model is used to process the overlapping speech segments corresponding to the time markers of the speech overlapping area to obtain the separated speech content of the target nursing staff.

[0116] In practical applications, the speech segments requiring speaker separation are first identified based on the time stamps of the overlapping speech regions obtained in step S1031. Each overlapping speech segment is input into a trained speaker-conditioning model, along with the target caregiver's speaker embedding vector as conditioning information. The model dynamically adjusts its internal parameters using the FiLM mechanism, focusing on extracting the target caregiver's speech content. Specifically, the model first converts the input speech into a time-frequency representation. Then, a conditionally controlled time-frequency masking mechanism is applied to enhance the target speaker's speech components while suppressing those from other speakers and background noise. Finally, the model converts the enhanced time-frequency representation back into a time-domain signal to obtain the separated target caregiver's speech content. Post-processing techniques, such as spectral subtraction and adaptive filtering, are also applied to further improve the quality of the separated speech. The separated speech content is converted to text using the speech recognition technology used in step S102, resulting in a transcript of the target speaker's speech in the overlapping region. This process enables accurate extraction of the target caregiver's speech content in complex multi-person conversation scenarios, providing reliable input for subsequent semantic analysis.

[0117] A dynamic speaker tracking mechanism is established to associate the voice segments of the same nurse in different time periods through continuous matching of voiceprint features to obtain the associated voice segments.

[0118] In real-world nursing scenarios, nurses may be interrupted mid-speech or alternate conversations with patients or other healthcare professionals. Therefore, dynamic tracking and association of dispersed speech content is necessary. First, a speaker tracking model is constructed that tracks the appearance and disappearance of specific speakers within a continuous speech stream. Based on a recurrent neural network architecture, the tracking model memorizes and updates speaker status information. For each speech segment, x-vector features are extracted and a similarity score is calculated with the target nurse's voiceprint template. An adaptive threshold mechanism dynamically adjusts the decision threshold based on ambient noise levels and voice quality to improve tracking robustness. Speech segments identified as belonging to the same nurse are labeled as related, and their temporal order and contextual relationships are recorded. Semantic coherence between speech segments is also analyzed to identify interrupted segments that may belong to the same narrative. For example, when a nurse is interrupted by a patient's question while describing a patient's condition and then continues describing the patient, the system can identify this situation and associate the two descriptions. This dynamic tracking mechanism enables accurate identification and association of different speech segments belonging to the same nurse in complex multi-person conversation environments, providing a foundation for subsequent speech integration.

[0119] Finally, the associated speech segments are integrated into a complete narrative based on time sequence and semantic coherence while retaining timestamp information to obtain the target speaker's speech transcription text.

[0120] In this step, the scattered speech segments need to be integrated into a coherent narrative to form a complete nursing record. First, the associated speech segments are arranged chronologically to establish a preliminary narrative framework. Then, natural language processing techniques are applied to analyze the semantic relationships between the segments and identify logically coherent narrative threads. Potential semantic gaps, such as ambiguous references and logical jumps, are detected and repaired. For example, when a nurse describes different symptoms of the same patient at different time points, these descriptions can be identified as belonging to the same topic and integrated into a coherent symptom description. Medical domain knowledge is also applied to assist in the integration process, such as understanding the standard sequence of nursing processes and identifying common nursing action sequences. The original timestamp information is retained for each integrated speech segment to facilitate later verification and tracing. Furthermore, operations during the integration process, such as segment merging and sequence adjustments, are recorded to ensure transparency and traceability of data processing. Through this integration process, the scattered speech segments are converted into a coherent target speaker's speech transcript, accurately reflecting the complete narrative of the nurse, providing high-quality input for subsequent semantic analysis and structured processing.

[0121] By implementing the five detailed steps described above, it is possible to accurately separate and identify the speech content of the target caregiver in complex medical environments and convert it into a coherent text transcript. This technical solution overcomes the limitations of traditional speech recognition in multi-speaker scenarios, significantly improving the accuracy and practicality of speech-to-text conversion in medical scenarios and laying a solid foundation for the generation of intelligent nursing records.

[0122] Step S104 includes the following sub-steps:

[0123] Step S1041: Using the pre-trained Wav2Vec2, HuBERT and WavLM self-supervised learning models, the acoustic features and semantic features of the target speaker's speech transcription text are extracted from different dimensions to obtain a multimodal feature representation.

[0124] In this step, based on the target speaker's speech transcription text output in step S103, three advanced self-supervised learning models are used to extract the acoustic and semantic features of the speech from different dimensions. The characteristic of the self-supervised learning model is that it can learn useful representations from a large amount of unlabeled data, without the need for a large amount of manually labeled training data. It is particularly suitable for speech processing rich in professional terminology in medical scenarios. First, the target speaker's speech transcription text is accurately time-aligned with the corresponding original speech signal to ensure the matching of text and speech. This alignment process uses a forced alignment algorithm to achieve accurate correspondence at the word level or phoneme level. Then, three pre-trained self-supervised learning models are deployed: Wav2Vec2, HuBERT and WavLM. These three models have their own strengths and can provide complementary feature representations. The Wav2Vec2 model is pre-trained using a contrastive learning method. It learns speech representation by predicting the masked speech part. It is particularly good at extracting low-level acoustic features and speech unit representations of speech, and has a high accuracy rate for phoneme-level recognition. The HuBERT model combines hidden units with a BERT-like training approach. It first uses a clustering algorithm to assign pseudo-labels to speech segments and then predicts pseudo-labels for the masked portions. This approach is particularly adept at capturing the phonetic and phonological characteristics of speech and has excellent representation of prosody and intonation. The WavLM model, an improvement on WaveNet, employs a denoising autoencoder architecture and a multi-task learning framework. It specializes in processing speech under various acoustic conditions, including complex scenarios like noise and overlapping speech, and is highly robust to the noisy speech found in medical environments. Speech signals are fed into these three models, and features are extracted from different layers of each model. Low-level features primarily contain acoustic information, such as basic acoustic features like phonemes, pitch, and energy; mid-level features contain more prosodic information, such as intonation, stress, and pauses; and high-level features are closer to semantic information, reflecting the semantic intent and emotional leaning of the speech content. For medical terminology, particular attention is paid to extracting pronunciation features and contextual relationships to improve subsequent recognition and comprehension accuracy. Through this multi-model feature extraction process, multimodal feature representations are obtained. These features not only retain the acoustic characteristics of the original speech, but also contain rich semantic and prosodic information, providing a comprehensive data basis for subsequent breathing block-level speech feature analysis.

[0125] The specific architectures and training methods of the three self-supervised learning models are as follows: The Wav2Vec2 model uses a Transformer-based encoder with 12 layers of transformer blocks, each with 8 attention heads, a model dimension of 768, and a feedforward neural network dimension of 3072. The model pre-training phase uses a mask prediction task, randomly masking approximately 15% of the speech representations and predicting the correct speech units from a quantized codebook. The HuBERT model also uses a 12-layer transformer structure, but its pre-training adopts a two-stage approach: first, a K-means clusterer (K=100) is trained using MFCC features to generate pseudo-labels, which are then used for mask prediction training. In the second stage, the hidden layer representations of the first-stage model are re-clustered to generate better pseudo-labels and further optimize the model. The WavLM model extends the HuBERT architecture, adding pre-training tasks for denoising and overlapping speech separation. It contains 24 layers of transformer blocks with a hidden dimension of 1024, and uses over 100,000 hours of diverse speech data in pre-training. All three models were fine-tuned on medical speech data, using approximately 2,000 hours of labeled medical conversation data. The fine-tuning used the Adam optimizer, a learning rate of 3e-5, a polynomial decay strategy after linear warmup, a batch size of 16, and 50 epochs.

[0126] Step S1042: Based on the Silero voice activity detection technology, the multimodal feature representation is divided into breathing blocks and block-level fluency indicators including speech rate, pause duration and repetition pattern are analyzed to obtain speech features with prosody markings.

[0127] In this step, the multimodal feature representation obtained in step S1041 is subjected to a more refined time scale analysis, particularly analyzing the prosodic features of the speech at the breathing chunk level. A breathing chunk refers to a speech segment completed by a speaker in one breath. It is a natural unit of speech analysis and is more in line with the natural rhythm of human speech than a fixed time window. Speech is finely divided based on Silero Voice Activity Detection (Silero-VAD) technology, which is an efficient voice activity detection tool that can accurately identify voiced and silent segments in speech. First, the pause points in the speech are detected, which usually correspond to the speaker's breathing or semantic boundaries. Then, according to the length and position of the pauses, the speech is naturally segmented into multiple breathing chunks. For each breathing chunk, a series of chunk-level fluency metrics are calculated: Speech Rate, which measures the number of syllables or words per second and reflects the speed of speech. Healthcare professionals often adjust their speech rate to emphasize key content when describing important information. Pause Duration, which analyzes the length of pauses within and between chunks to reflect the rhythm of speech and thought processes. Longer pauses may indicate the importance of information or the speaker's uncertainty. Stress Pattern, which identifies stressed syllables in speech and analyzes emphasized words or information points. Healthcare professionals often use stress to emphasize key medical terms or outliers. N-gram Repetition, which detects repetitive patterns in phrases or groups of words, may indicate hesitation, emphasis, or correction by the speaker. This may indicate information that requires special attention in medical records. Pitch Variation, which analyzes changes in the fundamental frequency of speech and reflects the speaker's emotion and tone. Sudden pitch changes may indicate surprise or attention to an unusual situation. The system also analyzes changes in the speech's energy profile and spectral centroid, which reflect speech clarity and expressive certainty. These fluency metrics are combined with speech content to form prosodic speech features. For example, when a nurse describes the important clinical finding "a patient is allergic to penicillin," they may slow down their speech, add emphasis, and pause before and after. These changes can be captured and marked as important information. This analysis process captures not only the content of the speech but also the implicit information contained in the way it is spoken, which is crucial for understanding the true intent and significance of medical speech.

[0128] Step S1043: The outputs of different self-supervised learning models are fused through a learnable weighting mechanism, the speech features with prosodic markers are combined, the contributions of acoustic and language features are balanced, and a comprehensive feature representation is obtained.

[0129] In this step, the multi-model features of step S1041 and the prosodic features of step S1042 are intelligently fused through a learnable weighting mechanism to form the optimal comprehensive feature representation. A feature fusion network based on the attention mechanism is designed, which can dynamically adjust the weights of different features according to the input content and task requirements. First, linear transformations are applied to the feature vectors from the three models Wav2Vec2, HuBERT, and WavLM, respectively, and they are mapped to the same dimensional space for subsequent fusion. Then, the attention score of each feature vector is calculated. This calculation process can be expressed as: ai = softmax(W·vi + b), where vi is the feature vector, W and b are learnable parameter matrices and bias terms, which are optimized by backpropagation. The attention score reflects the importance of each feature vector in understanding the current speech content. The feature vectors are weighted and summed according to the calculated attention scores to obtain the fused feature: vfused = Σ(ai·vi). This dynamic weighting mechanism adaptively adjusts feature weights based on different types of nursing content. For specialized terms like medication names, the HuBERT model features are weighted more heavily, leveraging their strengths in phonetics and phonology. For numerical information (such as body temperature and blood pressure), the Wav2Vec2 features are weighted more heavily, leveraging their ability to accurately identify them. For content recorded in noisy environments, the WavLM features are weighted more heavily, leveraging their strengths in handling complex acoustic environments. Furthermore, the weights of acoustic and linguistic features are balanced to ensure their complementarity. Acoustic features, primarily including phonemes, pitch, and energy, facilitate accurate recognition of specialized terms. Linguistic features, primarily including semantics and syntactic structure, contribute to understanding the overall meaning of the nursing content. Additional weight is assigned to the prosodic marker features obtained in step S1042, as these features often indicate important information. For example, when a slowing of speech rate, increased pauses, or increased stress is detected, the corresponding weight is increased, ensuring that important clinical information receives adequate attention in subsequent processing. The feature fusion network continuously optimizes its weight allocation strategy by training on labeled data, adapting to different nursing scenarios and the expressive habits of different nurses. This dynamic feature fusion process yields a comprehensive feature representation that balances the strengths of various models while preserving key information and prosodic features of the speech content, providing a comprehensive and rich feature foundation for further deep semantic analysis.

[0130] Step S1044: Applying a hierarchical CNN-BiLSTM network architecture to capture local and long-term dependencies, performing deep semantic analysis on the comprehensive feature representation, and obtaining the structured text with clinical semantic features.

[0131] In this step, the comprehensive feature representation obtained in step S1043 undergoes deep semantic analysis and structural processing, converting the unstructured spoken content into structured text with clinical semantic markers. A combined architecture of a layered convolutional neural network (CNN) and a bidirectional long short-term memory (BiLSTM) network is used for text processing. This architectural design leverages the advantages of CNNs in capturing local features and the ability of BiLSTMs to handle long-term dependencies, making it particularly suitable for processing the complex semantic relationships found in medical speech. At the bottom layer of the network, a multi-layer CNN is used to process the comprehensive feature representation of the input. The CNN convolution kernels vary in size, enabling the capture of local feature patterns across a wide range of domains. Small convolution kernels (e.g., 3×1) are used to capture word-level features, such as medical terminology and numerical expressions; medium-sized convolution kernels (e.g., 5×1) are used to capture phrase-level features, such as medical phrases like "intravenous infusion" and "high blood pressure"; and large convolution kernels (e.g., 7×1 or 9×1) are used to capture sentence-level contextual information. The output of the CNN layer undergoes max pooling and batch normalization before being input to the BiLSTM layer. The BiLSTM layer, composed of multiple layers of bidirectional LSTM units, can simultaneously consider both forward and backward information in a sequence and capture long-range dependencies. This is crucial for understanding causal, temporal, and logical relationships in medical speech. For example, when a nurse describes a patient experiencing pain, and after administering 50mg of pethidine intramuscularly, the pain was relieved, it is important to understand that "pain relief" is a result of administering pethidine. This long-range dependency can be effectively captured by the BiLSTM. The network's layered design enables simultaneous processing of language units at different granularities: at the character / word level, identifying basic elements such as medical terms and numerical values; at the phrase level, understanding complex expressions such as medical procedures and symptom descriptions; at the sentence level, grasping complete nursing actions or observations; and at the paragraph level, understanding the overall nursing process and logical relationships. At each layer, the following processing is performed: entity recognition, identifying medical entities in the text, such as medications, symptoms, and signs; relationship extraction, analyzing relationships between entities, such as "drug-dose" and "symptom-site"; temporal reasoning, understanding the temporal sequence and duration of events; and logical inference, inferring implicit clinical information based on medical knowledge. The final layer of the network uses a conditional random field (CRF) for sequence annotation, assigning clinical semantic labels to each element in the text, such as "drug name," "dose," "route of administration," "symptoms," and "signs." The resulting structured text, with clinical semantic features, contains the following elements: nursing procedure type (e.g., medication, vital sign monitoring, wound management); procedure object (e.g., specific medication, monitoring item); numerical parameters (e.g., dosage, temperature reading); temporal information (e.g., procedure time, duration); observations (e.g., patient response, wound status); and importance tags (e.g., important information tagged based on prosodic features and medical knowledge).Through this deep semantic analysis process, unstructured oral content is converted into structured text with rich semantic tags, laying the foundation for subsequent clinical information extraction driven by the medical ontology knowledge base.

[0132] The architecture of the hierarchical CNN-BiLSTM network is as follows: the first layer consists of three parallel one-dimensional CNNs with convolution kernel sizes of 3, 5, and 7, respectively. Each branch contains 128 convolution kernels, capturing local features at different scales. The CNN outputs undergo max pooling and concatenation before passing through batch normalization and a dropout layer (with a ratio of 0.3). The middle layer of the network consists of three BiLSTM layers, each with 256 hidden units (128 in each direction). Residual connections are added between layers to prevent the vanishing gradient problem. An attention mechanism is used after each BiLSTM layer to aggregate sequence information, with attention weights calculated using a learnable query vector. The top layer of the network is a conditional random field (CRF) layer, which models transition probabilities between labels and improves the consistency of sequence annotations. The network is trained using a combined loss function: L = λ1LCRF + λ2Latt, where LCRF is the CRF negative log-likelihood loss, Latt is the attention-guided loss, and λ1 and λ2 are weight coefficients (set to 1.0 and 0.3, respectively). The model was trained using a medical semantic annotation dataset, consisting of approximately 5,000 annotated nursing records covering 20 clinical semantic categories. Training was performed using the Adam optimizer with an initial learning rate of 0.001 and an early stopping strategy, stopping training when the F1 score on the validation set showed no improvement for five consecutive epochs.

[0133] Step S105 includes the following sub-steps:

[0134] Step S1051: Based on clinical nursing standards and professional guidelines, a multi-dimensional medical ontology knowledge base including nursing operations, drugs, symptoms, and signs is constructed to obtain a structured medical ontology knowledge model.

[0135] In this step, a comprehensive medical care ontology knowledge base was constructed to serve as the knowledge foundation for extracting and standardizing clinical information. An ontology knowledge base is a formalized knowledge representation method that defines concepts, attributes, and relationships within a domain and supports complex reasoning and querying. First, a variety of clinical nursing standards and professional guidelines were collected and organized, including nursing standards issued by the National Health Commission, nursing practice standards of the International Council of Nursing (ICN), nursing guidelines for various specialties, and internal hospital nursing procedures. Then, a team of medical experts and knowledge engineers used ontology engineering methods to construct the medical care ontology knowledge base. The knowledge base construction followed a combined top-down and bottom-up approach: the top-down approach defined key concept categories and hierarchies, such as top-level concepts such as nursing procedures, medications, symptoms, and signs; the bottom-up approach extracted specific instances and relationships from actual nursing records to enrich the knowledge base content. The medical nursing ontology knowledge base includes the following key components: a conceptual hierarchy, such as a drug classification system (antibiotics, analgesics, antihypertensive drugs, etc.), a symptom classification system (pain, fever, dyspnea, etc.), a sign classification system (blood pressure, pulse, respiration, etc.), and a nursing procedure classification system (drug administration, dressing changes, circuit care, etc.); entity attribute definitions, such as drug dosage form, usage, contraindications, adverse reactions, symptom severity, duration, and triggers, and normal ranges and abnormal classifications of physical signs; relationship networks, such as "drug-indication" relationships, "symptom-disease" relationships, and "nursing procedure-applicable situation" relationships; and rule constraints, such as drug dosage ranges, dosing intervals, standard nursing procedures, and frequency requirements for physical sign monitoring. The knowledge base is represented using the Web Ontology Language (OWL) to support complex semantic queries and reasoning. Furthermore, it incorporates existing medical standard terminology, such as SNOMEDCT (Synthetic Medical Terminology - Clinical Terminology), ICD-10 (International Classification of Diseases, 10th Revision), and LOINC (Logical Observation Identifier Names and Codes), to ensure that concepts in the knowledge base are consistent with international standards. Through this process, a structured medical ontology knowledge model was constructed. This model not only contains rich medical knowledge, but also defines the semantic relationship between concepts, providing strong knowledge support for subsequent clinical information extraction and standardized processing.

[0136] Step S1052: Utilize the structured medical ontology knowledge model to perform entity recognition and relationship extraction on the structured text with clinical semantic features, map the descriptive language in the structured text with clinical semantic features to standard medical terms, and obtain a standardized clinical terminology set.

[0137] In this step, the medical ontology knowledge model constructed in step S1051 is used to further process the structured text with clinical semantic features output from step S104, converting non-standard descriptive language into standardized medical terminology. First, medical entity recognition is performed to identify medical entities in the text, such as drug names, symptom descriptions, physical sign values, and nursing procedures. This recognition process combines rule-based and machine learning methods: the rule-based approach uses a medical terminology dictionary and regular expressions to match medical terms in the text; the machine learning approach uses a conditional random field (CRF) or a bidirectional long short-term memory network (BiLSTM) combined with a conditional random field model (BiLSTM-CRF) for sequence labeling, identifying medical entities and their types in the text. Then, term standardization is performed to map the identified medical entities to standard concepts in the medical ontology knowledge base. This mapping process handles a variety of non-standard expressions: synonym processing, unifying synonymous expressions into standard expressions, such as converting "intravenous drip," "intravenous drip," and "IV" into the standard "intravenous infusion"; abbreviation expansion, expanding medical abbreviations into complete expressions, such as expanding "BP," "T," "P," and "R" into "blood pressure," "temperature," "pulse," and "respiration"; numerical unit standardization, unifying various numerical expressions into standard units and formats, such as converting "37.5 degrees Celsius" into "37.5°C" and "one hundred and twenty / eighty" into "BP 120 / 80 mmHg"; and fuzzy expression refinement, converting ambiguous descriptions into precise medical expressions, such as standardizing "blood sugar 8.0 mmol / L" into "blood sugar 8.X mmol / L" and converting "mild redness and swelling" into "mild redness and swelling (grade I)." Relationship extraction also identifies semantic relationships between medical entities, such as "drug-dose," "symptom-site," and "procedure-time." Relationship extraction utilizes a strategy that combines rule-based methods based on dependency parsing with deep learning-based approaches to accurately identify entity relationships within complex text. Identified relationships are mapped to standard relationship types defined in the medical ontology knowledge base. This process transforms non-standard, diverse descriptive language into standardized, unified medical terminology, forming a standardized clinical terminology set. This standardization process ensures the accuracy and consistency of medical information, laying the foundation for subsequent key information extraction and record generation.

[0138] Step S1053: Apply a deep learning key information extraction algorithm to automatically identify and rate the important clinical information in the standardized clinical terminology set to obtain a clinical information set with importance marks.

[0139] In this step, a deep learning-based key information extraction algorithm is applied to automatically identify and rank important clinical information within a standardized clinical terminology set. First, a bidirectional long short-term memory network model based on an attention mechanism is constructed to semantically encode clinical terms. This model utilizes a multi-layer BiLSTM architecture, which can capture long-range dependencies in sequential data and is particularly well-suited for processing contextual information in medical texts. The model also integrates a multi-head self-attention mechanism to automatically learn the strength of associations between different positions and highlight important information. The standardized clinical terminology set is input into the model to generate context-aware semantic vector representations for each term. These vector representations not only contain the semantic information of the term itself but also incorporate information about its context within the entire nursing record, providing a more comprehensive reflection of the term's clinical significance. Next, a conditional random field model is used to sequentially annotate the semantic vector representations of the terms and identify key clinical entity types. The conditional random field model considers transition probabilities between labels and generates a globally optimal annotation sequence, avoiding the annotation inconsistencies that can result from independent classification. A series of key clinical entity types, such as "abnormal signs," "serious symptoms," "adverse drug reactions," and "high-risk procedures," are predefined, and each type is assigned a different weight to reflect its clinical importance. Next, a graph convolutional neural network is applied to analyze the dependencies and semantic associations between entities in the entity annotation results. The graph convolutional neural network constructs an entity relationship graph (ERG) using clinical entities as nodes and relationships between entities as edges. Through multi-layer graph convolution operations, it can capture high-order associations between entities and discover potentially complex medical patterns, such as the causal relationship between drugs and symptoms and the temporal trends of symptom changes. This graph-based representation method is particularly well-suited for processing complex relational networks in medical data. Based on the knowledge of medical experts and clinical guidelines, an importance scoring model is constructed to quantitatively assess the importance of clinical information in the ERG. The model comprehensively considers multiple factors: the weight of clinical entity type, such as abnormal signs generally being more important than normal signs; the degree of abnormality, such as values ​​significantly deviating from the normal range being more important than minor deviations; temporal factors, such as newly emerging symptoms being more worthy of attention than persistent symptoms; relationship strength, such as direct causal relationships being more important than indirect associations; and medical knowledge, such as the likelihood that certain symptom combinations may indicate serious medical conditions. The model outputs a significance score for each piece of clinical information, reflecting its importance in medical decision-making. Finally, based on the significance score, clinical information in the entity-relationship diagram is assigned priority tags and urgency indicators. Information is categorized into four levels: "routine," "needing attention," "important," and "urgent." Different visual markers (such as colors and icons) are used to highlight important information in the final nursing record. This process results in a set of clinical information with significance tags. These tags will guide the subsequent generation of nursing records, ensuring that key information is highlighted and prioritized, thereby enhancing the clinical value and practicality of nursing records.

[0140] In a preferred embodiment, a deep learning key information extraction algorithm is applied to automatically identify and rate the important clinical information in the standardized clinical terminology set, thereby obtaining a clinical information set with importance tags, including:

[0141] A bidirectional long short-term memory network model based on the attention mechanism is constructed to semantically encode the clinical terms in the standardized clinical term set to obtain a semantic vector representation of the terms.

[0142] The core of this step is to convert standardized clinical terms into computer-processable numerical vectors while preserving the terms' semantic information and contextual relationships. A multi-layer BiLSTM (Bidirectional Long Short-Term Memory) architecture serves as the underlying framework. This architecture simultaneously considers both forward and backward information in a sequence, making it particularly well-suited for capturing long-range dependencies in medical text. For example, when describing content such as "the patient's blood pressure was normal upon admission, but began to rise the day after surgery," the BiLSTM can correlate long-range changes in blood pressure. First, each clinical term is converted into a pretrained medical domain word vector (such as one generated by Medical Word2Vec or BioBERT) as input to the BiLSTM. The BiLSTM network consists of 3-5 layers, each with 256 hidden units, capable of extracting increasingly abstract semantic features layer by layer. A multi-head self-attention mechanism is integrated within the BiLSTM, enabling the model to automatically learn the strength of associations between different positions in the sequence. The multi-head attention mechanism projects the query (Q), key (K), and value (V) into multiple subspaces, independently calculating attention in each subspace and then merging the results. This enables the model to simultaneously focus on different types of semantic relationships. For example, one attention head might focus on the relationship between symptoms and signs, while another might focus on the relationship between medication and dosage. Residual connections and layer normalization are also introduced to improve the stability and convergence speed of model training. Through this complex neural network architecture, each clinical term and its context are mapped into a 768-dimensional semantic vector representation. These vectors not only encode the semantic information of the term itself but also incorporate information about its context within the entire nursing record, enabling the distinction of subtle semantic differences between the same term in different contexts. For example, the vector representation of the word "fever" in the contexts of "patient with mild fever" and "patient with persistent high fever" will differ significantly, reflecting its different clinical importance.

[0143] The conditional random field model is used to perform sequence annotation on the semantic vector representation of the term to identify key clinical entity types and obtain entity annotation results.

[0144] Conditional Random Fields (CRFs) are a probabilistic graphical model specifically designed for sequence labeling. They consider transition probabilities between labels and generate globally optimal labeling sequences. This property makes them particularly well-suited for entity recognition in medical text, as medical entities often follow specific patterns of occurrence and combination. The term semantic vectors obtained in the previous step are mapped to the label space through a fully connected layer, yielding preliminary scores for each term corresponding to each entity type. The CRF layer then considers transition probabilities between adjacent labels to calculate the optimal labeling path for the entire sequence. A set of key clinical entity types is predefined, including basic and complex types. Basic types include: vital signs (such as blood pressure, temperature, pulse, respiration, and blood oxygen saturation), symptoms (such as pain, nausea, and dyspnea), medication information (drug name, dosage, and route of administration), nursing procedures (such as venipuncture, catheter care, and wound dressing changes), examination results (such as blood count and imaging findings), and assessment results (such as pain scores and pressure ulcer risk scores). Composite categories include: abnormal signs (such as hypertension and hypoxemia), severe symptoms (such as severe pain and respiratory distress), adverse drug reactions (such as allergic reactions and drug-related rashes), high-risk procedures (such as deep vein puncture and endotracheal intubation), and critical examination abnormalities (such as electrolyte imbalance and coagulation abnormalities). Each entity type is assigned a different weight to reflect its clinical importance. For example, "severe symptoms" has a higher weight than "common symptoms," and "abnormal signs" has a higher weight than "normal signs." The CRF model outputs the most likely entity type for each term, forming a complete entity annotation sequence. This annotation process not only identifies the boundaries and types of clinical entities but also provides a preliminary reflection of their clinical importance through pre-set weights for entity types, laying the foundation for subsequent relationship analysis and importance assessment.

[0145] A graph convolutional neural network is used to analyze the dependencies and semantic associations between entities in the entity annotation results to obtain an entity relationship graph.

[0146] The importance of medical information depends not only on the entities themselves but also on the network of relationships between them. For example, a single symptom of "headache" may not be serious, but if it occurs simultaneously with "high blood pressure" or "blurred vision," it may indicate a more serious medical condition. Graph Convolutional Neural Networks (GCNs) are deep learning models specialized for processing graph-structured data, effectively capturing complex relationships between nodes. First, an initial entity-relationship graph is constructed, with the identified clinical entities as nodes and the relationships between entities as edges. Node features are initialized to semantic vector representations of the entities, and edge types are determined based on predefined medical relationship types, including temporal relationships (sequential, simultaneous), causal relationships (cause, induce, alleviate), part-whole relationships (belong to, contain), degree relationships (exacerbate, alleviate), and location relationships (located in, near). Various approaches are used to identify relationships between entities: rule-based methods use predefined grammatical patterns and keywords to identify common relationships; dependency parsing methods analyze the syntactic structure of sentences to identify grammatical relationships between entities; and deep learning methods use relationship extraction models to learn entity relationships directly from text. After constructing the initial graph, a multi-layer graph convolutional network is applied to the graph. The computational process for each GCN layer can be expressed as: H(l+1) = σ(D(-1 / 2)ÂD(-1 / 2)H(l)W(l)), where H(l) is the node feature matrix of layer l, Â is the adjacency matrix with self-loops added, D is the degree matrix, W(l) is the learnable weight matrix, and σ is the nonlinear activation function. Through multi-layer graph convolution operations, the features of each node not only contain its own information but also incorporate information from its neighboring nodes, thereby capturing high-order connections between entities. A three-layer GCN is used, with a hidden layer dimension of 256 and a Reluctant Unit (ReLU) activation function. Furthermore, an attention mechanism is introduced into the graph convolution, enabling the model to distinguish the importance of different neighboring nodes. After processing, enhanced node and edge features are obtained, which encode the position and importance of entities in the entire relationship network. Ultimately, a complete entity-relationship graph is generated, containing rich node attributes (such as entity type and semantic features) and edge attributes (such as relationship type and relationship strength). This graph structure representation enables the analysis of the structure and importance of clinical information from a holistic perspective, and the discovery of potential complex medical patterns, such as symptom combinations, trends in physical sign changes, and treatment response patterns.

[0147] An importance scoring model is constructed based on medical expert knowledge and clinical guidelines to quantitatively evaluate the importance of the clinical information in the entity relationship diagram to obtain an importance score.

[0148] Assessing the importance of clinical information is a complex task that requires comprehensive consideration of multiple medical factors and contextual information. The constructed importance scoring model combines rule-based and machine learning methods to assess the importance of clinical information from multiple dimensions. First, a series of scoring rules based on medical knowledge were defined. These rules were developed by experienced nursing experts and clinicians and cover multiple scoring dimensions: The type weight of clinical entities: Different types of clinical entities have different baseline importance, e.g., abnormal signs are generally more important than normal signs, and severe symptoms are more worthy of attention than mild ones; the degree of abnormality: The degree to which a numerical indicator deviates from the normal range directly affects its importance. Standardized deviation scores, such as those for blood pressure and blood sugar, are calculated based on medical reference ranges; the time factor: Newly emerging symptoms or changes in signs are generally more concerning than long-standing conditions. Time stamps in clinical information are analyzed to identify emerging events and changing trends; relationship strength: Entities with more connections or stronger associations in the entity-relationship graph are generally more important. This factor is quantified by calculating node centrality metrics (such as degree centrality and eigenvector centrality); and medical knowledge: Certain symptom combinations or sign patterns may indicate serious medical conditions. A library of "alert combinations" containing clinical manifestation combinations that require special attention is maintained. Next, these rules are used to calculate a preliminary importance score for each piece of clinical information, which is then fed into an ensemble learning model for final scoring. This ensemble model combines gradient boosted decision trees (such as XGBoost) with deep neural networks, enabling it to learn complex, nonlinear scoring patterns. The model is trained on a large amount of labeled data, including clinical information from nursing records and their importance ratings annotated by healthcare professionals. Through this training process, the model learns implicit rules for assessing the importance of clinical information in different healthcare scenarios. In practice, each piece of clinical information in the entity-relationship graph (ERG), including the entity itself and its relationships, is converted into a feature vector and fed into the trained scoring model to generate an importance score from 0 to 100. This score reflects the importance of the information in medical decision-making, with higher scores indicating greater importance and requiring greater attention from healthcare professionals. The scoring model parameters are also dynamically adjusted based on different departments and care scenarios to adapt to the clinical priorities of different specialties. For example, in a cardiology department, abnormalities in heart rate and blood pressure may receive higher importance scores, while in a respiratory department, abnormalities in oxygen saturation and respiratory rate may be more significant.

[0149] Priority marks and urgency marks are added to the clinical information in the entity relationship diagram according to the importance score to obtain the clinical information set with importance marks.

[0150] This step converts the numerical importance score into an intuitive marker, making it easier to highlight important information in the final nursing record. First, clinical information is divided into four levels based on the importance score: Regular (score 0-25): routine clinical information that does not require special attention, such as vital signs within the normal range and daily nursing operations; Attention (score 26-50): information that requires the attention of nursing staff but is not urgent, such as slightly abnormal test results and symptoms that require follow-up; Important (score 51-75): important clinical information that requires timely treatment or close monitoring, such as obvious abnormal signs and moderate symptoms; Urgent (score 76-100): urgent clinical information that requires immediate intervention or treatment, such as life-threatening abnormal signs and serious adverse reactions. Then, different visual markers were designed for each level to distinguish them in the final nursing record: Routine information uses a standard font and color, with no special markings; information requiring attention uses a blue font or a blue asterisk to indicate it requires attention; important information uses an orange font or an orange exclamation mark to indicate importance; and urgent information uses a red font or a red double exclamation mark to indicate urgency. In addition to basic priority markings, specific urgency markers are added to provide more detailed clinical information: time-sensitive markers indicate that information requires action within a specific time window, such as "review within 4 hours" or "report immediately"; trend markers indicate the changing trend of indicators, such as "↑" for an increase, "↓" for a decrease, and "↑↑" for a significant increase; and intervention recommendation markers provide preliminary intervention recommendations based on clinical guidelines, such as "consider fluid replacement" and "oxygen therapy required." The specific format of the markers is also adjusted according to the specific requirements of the hospital and the display capabilities of the electronic medical record to ensure that the markers are clearly visible in practice and do not affect the overall readability of the record. Through this tagging process, importance scores are converted into intuitive visual markers, enabling caregivers to quickly identify important information within the record, improving the efficiency and safety of nursing care. These markers also facilitate information exchange among medical team members, ensuring that critical clinical information is not overlooked. Ultimately, a set of clinical information with importance markers is output. These sets of information become the core content for subsequent nursing record generation, ensuring that the resulting records not only fully document the nursing process but also highlight important information requiring attention.

[0151] Constructing a bidirectional long short-term memory network model based on an attention mechanism to semantically encode clinical terms in the standardized clinical term set to obtain a semantic vector representation of the term;

[0152] Using a conditional random field model to perform sequence annotation on the semantic vector representation of the term to identify key clinical entity types and obtain entity annotation results;

[0153] Applying a graph convolutional neural network to analyze the dependencies and semantic associations between entities in the entity annotation results to obtain an entity relationship graph;

[0154] An importance scoring model is constructed based on medical expert knowledge and clinical guidelines to quantitatively evaluate the importance of the clinical information in the entity relationship diagram to obtain an importance score;

[0155] Priority marks and urgency marks are added to the clinical information in the entity relationship diagram according to the importance score to obtain the clinical information set with importance marks.

[0156] Step S1054: Classify and organize the clinical information set with importance marks into element groups that conform to the nursing record structure according to the element model of the nursing record specification to obtain the standardized nursing record element set.

[0157] In this step, the importance-tagged clinical information sets obtained in step S1053 are categorized and organized according to the standard element model for nursing records, forming element groups that conform to the nursing record structure. Multiple nursing record formats are supported, including the internationally accepted SOAP format (Subjective, Objective, Assessment, Plan), the DAR format (Data, Action, Response), and hospital-defined specialized nursing record formats. First, determine the record format to use based on the current nursing scenario and hospital settings. For the SOAP format, clinical information is categorized into the following categories: Subjective data (Subjective), including subjective information such as the patient's chief complaint, symptom description, and comfort evaluation, such as "The patient complained of pain in the right lower limb, with a VAS score of 7"; Objective data (Objective), including objective data such as physical sign measurements, examination results, and observations, such as "Body temperature 38.5°C, blood pressure 135 / 85 mmHg, and the wound dressing is clean and dry"; Assessment information (Assessment), including the nurse's assessment of the patient's condition and professional opinions such as risk assessment, such as "Considering postoperative incision pain, there are no signs of infection"; Plan content (Plan), including action plans such as treatment plans, nursing measures, and follow-up arrangements, such as "Give 50 mg of tramadol orally as prescribed by the doctor, and reassess the pain after 30 minutes." In the DAR format, information is divided into: Data, which includes a combination of subjective and objective data, such as patient symptom and sign measurements; Action, which includes implemented nursing measures and interventions, such as medication administration, dressing changes, and health education; and Response, which includes the patient's response to the nursing measures and their evaluation of their effectiveness, such as "pain relief after medication, VAS score reduced to 3." Specialty nursing records are customized based on the specific requirements of each specialty. For example, intensive care records may require more detailed vital sign monitoring and ventilator parameters; obstetric nursing records may include specific content such as fetal heart rate monitoring and labor observation. Other necessary elements are also added based on the hospital's specific nursing record requirements, such as standardized information such as basic patient information (name, age, gender, hospitalization number, etc.), recording time, and nurse signature. During the categorization process, the importance mark added in step S1053 is retained to ensure that important information is highlighted in the final nursing record. Furthermore, each element category is checked for completeness. If a category is found to be missing necessary information, a prompt will be generated, suggesting that the nurse supplement the relevant content. Through this classification and organization process, scattered clinical information is integrated into structured nursing record element sets. These element sets meet the format requirements of standard nursing records, contain complete nursing content, and retain the importance marks of information, providing a complete content basis for subsequent record generation.

[0158] Step S1055: continuously learning and updating the medical ontology knowledge model, optimizing the structure of the medical ontology knowledge base according to newly added clinical cases and nursing records, and obtaining a dynamically updated medical ontology knowledge model.

[0159] In this step, a continuous learning and updating mechanism for the medical ontology knowledge model is established, enabling the knowledge base to be continuously optimized and expanded with new clinical cases and nursing records. This mechanism ensures that the knowledge base can adapt to the development and changes in medical practice, maintaining its practicality and timeliness. First, a new term collection mechanism is established to automatically collect new medical terms that have not been recognized or processed in actual applications. When unknown terms are encountered, they are marked and recorded in a candidate term base, and their contextual information is preserved to facilitate subsequent analysis. These candidate terms are regularly reviewed by medical experts to confirm their medical significance and standard expression before being added to the knowledge base. Frequently occurring terms that have not yet been included are given higher priority to accelerate the review process. Then, a relationship discovery mechanism analyzes a large number of nursing records to discover and validate new entity relationship patterns. Using techniques such as association rule mining and frequent pattern mining, entity co-occurrence patterns are extracted from nursing records, such as the association between certain symptoms and specific diseases, or the association between certain medications and specific adverse reactions. After statistical verification and expert confirmation, these discovered relationship patterns are added to the relationship network of the knowledge base, enriching the semantic associations of the knowledge base. Concept hierarchical adjustments are also performed, adjusting the hierarchy and classification system of concepts in the knowledge base based on actual usage. For example, when a particular drug class is found to require more detailed classification in practical applications, recommendations are made to adjust the classification hierarchy of that class. When certain concepts are found to be frequently confused in practical use, recommendations are made to optimize the definitions and distinction criteria of these concepts. Furthermore, rule optimization is performed, optimizing and adjusting the parameters and constraints of knowledge rules based on feedback from actual applications. For example, drug dosage ranges are updated based on new clinical guidelines, standard process steps are adjusted based on actual nursing practices, and normal ranges for physical signs are updated based on the latest research findings. Knowledge graph expansion is also implemented, integrating newly discovered medical knowledge into the existing knowledge graph to expand its coverage. This includes adding new drugs, treatments, nursing techniques, and their relationships with existing concepts. A semi-automated update approach, combining machine learning algorithms and expert review, ensures the accuracy and reliability of knowledge updates. Machine learning algorithms identify potential knowledge updates from large amounts of data, while medical experts verify these findings and decide whether to incorporate them into the knowledge base. A knowledge version control mechanism has also been established to record every update to the knowledge base, enabling backtracking and comparison of knowledge content across different versions, ensuring the traceability and stability of knowledge updates. Through this continuous learning and updating mechanism, the medical ontology knowledge model can continuously adapt to new medical practices and terminology changes, maintaining its practicality and timeliness, and providing continuously updated knowledge support for the generation of intelligent nursing records.

[0160] Step S106 includes the following sub-steps:

[0161] Step S1061: Build a template library covering different nursing scenarios based on the hospital's existing nursing record specifications and electronic medical record standards to obtain a nursing record template set.

[0162] In this step, we build a comprehensive library of nursing record templates to serve as the foundation for automatically generating standardized nursing records. We first collect and analyze the hospital's existing nursing record standards and format requirements, including established nursing documentation standards, nursing quality control standards, and technical specifications for electronic medical records. We also reference standards such as the "Basic Specifications for Electronic Medical Records" and "Functional Specifications for Electronic Medical Records" to ensure that the generated nursing records meet industry standards. Furthermore, we collaborate closely with the institution's nursing department and involve senior head nurses and specialist nurses in template design to ensure the professionalism and practicality of the template content. The constructed template library covers various types of nursing record templates to meet the needs of different nursing scenarios: basic nursing record templates, used to record routine nursing content, such as vital signs monitoring, basic nursing operations, general patient condition, etc., suitable for daily nursing records of all departments; specialist nursing record templates, customized for the special needs of different specialties, such as the cardiology template focusing on cardiac function monitoring and electrocardiogram changes, the neurology template focusing on consciousness state and neurological assessment, and the oncology template focusing on chemotherapy response and pain management, etc.; special nursing operation templates, used to record the detailed process of specific nursing operations, such as intravenous puncture, catheter care, wound dressing change, endotracheal intubation care, etc., including complete content such as pre-operation preparation, operation process, and post-operation observation; assessment record templates, used to record various professional assessments, such as pain assessment, pressure ulcer risk assessment, fall risk assessment, nutritional status assessment, etc., including assessment tools, scoring standards and intervention recommendations; shift handover record templates, used for information transmission when nursing staff hand over shifts, including key patient conditions, special treatment arrangements, to-do items, etc. Each template is carefully designed, defining a specific structure and fields, including mandatory, optional, and automatically calculated fields. Mandatory fields are core content that must be included in a record, such as basic patient information, recording time, and key nursing content. Optional fields are optional content, such as special observations and additional nursing measures, that can be entered based on specific circumstances. Automatically calculated fields are automatically generated based on other fields, such as a total score calculated from individual scores or an average calculated from multiple measurements. Furthermore, content validation rules are defined for each template, such as checks for value ranges, mandatory field completeness, and logical consistency, to ensure that generated records conform to medical logic and documentation standards. Template definitions are stored in XML or JSON format for easy management and updating. Each template includes metadata, such as applicable department, applicable scenario, creation date, and version number, to facilitate selection of the appropriate template in real-world applications. This step establishes a structured, standardized nursing record template library, providing a format foundation and content framework for subsequent automated record generation.

[0163] Step S1062: Based on the content features and context information of the standardized nursing record element set, a template matching algorithm is applied to automatically select the most suitable nursing record template to obtain a matching record template.

[0164] In this step, the most appropriate template for the current nursing record element set is selected from the template library constructed in step S1061. This selection process is crucial, as different templates are suitable for different nursing scenarios and content types. Choosing the right template ensures that the generated nursing record is standardized and complete. First, the content characteristics of the standardized nursing record element set are analyzed, including element type distribution, keyword frequency, and terminology categories. Text classification techniques are used to preliminarily categorize the element set, identifying its primary content types, such as routine nursing records, specialized nursing records, and special procedure records. This classification process utilizes a pretrained medical text classification model, trained on a large number of annotated nursing records, to accurately identify different types of nursing content. Next, the contextual information of the nursing record element set is analyzed, including department information, patient profile, and nursing time points. This information can be obtained from hospital data or extracted from speech transcriptions. Department information directly influences template selection. Different departments often have specific nursing record requirements. Patient profiles, such as age and primary diagnosis, influence the focus of the record. Nursing time points, such as the day of admission, postoperatively, and before discharge, also require different recording emphases. Based on content features and contextual information, a template matching algorithm is applied to select the most appropriate template for the current record. This algorithm comprehensively considers multiple matching factors: content type matching, which evaluates the degree of match between the content type of the feature set and the applicable type of the template, such as whether postoperative care content should match the postoperative care template; keyword matching, which calculates the overlap between the keywords in the feature set and the keyword sets of each template, using TF-IDF weighted similarity scores; completeness assessment, which assesses the coverage of the required fields of each candidate template by the feature set, prioritizing templates with high field coverage; context analysis, which considers contextual factors such as department, patient condition, and nursing time point, prioritizing templates appropriate to the current context; and historical reference, which analyzes templates used in previous nursing records for the same patient and maintains template consistency where appropriate to facilitate continuity comparison of records. A comprehensive matching score is calculated for each candidate template, and the template with the highest score is selected as the final matching result. The matching score is calculated as follows: Score = w1ContentTypeMatch + w2KeywordSimilarity + w3FieldCoverage + w4ContextMatch + w5×HistoricalConsistency, where w1 through w5 represent the weights of each factor and can be adjusted based on the hospital's specific needs. To improve matching accuracy, machine learning methods are also used to optimize the matching algorithm. Nurses' feedback on template matching results is collected to construct a training dataset, and a supervised learning method is used to train the template recommendation model, continuously improving the performance of the matching algorithm. In practice, the top 3-5 templates are recommended, along with the matching reasons, for the nurse to select or confirm.This human-machine collaborative approach leverages the efficiency of the algorithm while retaining the judgment of professionals, ensuring the accuracy of template selection. Through this step, the most appropriate record template can be selected for the current set of nursing record elements, providing a structured framework for subsequent record generation.

[0165] Step S1063: Intelligently fill in the clinical information in the standardized nursing record element set according to the matching record template, and add metadata such as timestamp, operation type and record status to obtain a preliminarily formatted nursing record.

[0166] In this step, the clinical information in the standardized nursing record element set output in step S105 is intelligently populated according to the record template selected in step S1062 to form a structured nursing record. This process is similar to placing scattered pieces of information into a predefined framework, ensuring that each piece of information is properly positioned to form a complete record that complies with the standard. First, field mapping is performed to map each piece of information in the nursing record element set to the corresponding field in the template. This mapping process is based on semantic matching of fields, using a medical ontology knowledge base and semantic similarity calculation to determine the correspondence between elements and template fields. For example, "Body temperature 38.5°C" is mapped to the "Body temperature" field in the template, and "Intravenous infusion 500ml of normal saline" is mapped to the corresponding subfield of the "Infusion record" field. For complex nursing procedures, multiple related elements are mapped to the complete structure of the operation record template. For example, elements such as "Puncture site," "Puncture needle model," and "Infusion rate" are mapped to the various subfields of the "Intravenous infusion" operation record. Automatic calculation and populating of derived values ​​are then performed. Certain fields in the template require calculations or derivations based on the values ​​of other fields, and these calculations are performed automatically. For example, the average value is calculated based on multiple physical sign measurements, the total score is calculated based on the scores of each item on the pain rating scale, and the unit dose is calculated based on weight and dosage. Format conversion is also performed to ensure that the filled content meets the format requirements of the template field. This includes numerical formatting (such as retaining specific decimal places), date and time standardization (such as the unified use of a 24-hour clock), and text format normalization (such as the standard writing method of professional terminology). For required fields in the template, if there is no corresponding information in the nursing record element set, a default value will be filled in or marked as pending, and a prompt for supplementation will be issued during subsequent quality checks. Cross-validation is also performed to check the consistency between related fields, such as the matching of drug dosage and administration method, and the consistency of symptom description and assessment results. At the same time, rich metadata is added to the nursing record to enhance the traceability and management value of the record. This metadata includes: timestamp information, including record creation time, modification time, and event occurrence time; operation type identifier, indicating the nature of the record, such as routine record, special operation record, and evaluation record; record status marker, indicating the record's current status, such as draft, pending review, and completed; and identifiers, including the record generation method, template version used, and the percentage of automatically populated content. This metadata not only facilitates record management and querying but also enhances the legal validity and traceability of the records. The importance marker added in step S1053 is also retained, highlighting high-priority information during the filling process, such as using different colors or adding special markers, to ensure that important information is highlighted in the final record. Through this intelligent filling process, the dispersed set of nursing record elements is transformed into a structured, pre-formatted nursing record that conforms to the format requirements defined by the template and contains complete clinical information and necessary metadata.

[0167] Step S1064: Use medical common sense verification rules and integrity check algorithms to perform quality checks on the initially formatted nursing records, identify possible missing items and logical errors, and obtain intelligent completion suggestions.

[0168] In this step, a comprehensive quality check is performed on the preliminary formatted nursing record generated in step S1063 to ensure the completeness, accuracy and logical consistency of the record. This quality check process is similar to the review of nursing records by an experienced head nurse, which can identify potential problems and provide improvement suggestions. First, a completeness check is performed to verify whether the record contains all necessary information. According to the list of required fields defined in the template, check whether each required field has been filled in, such as patient basic information, recording time, main nursing content, etc. For required fields that are not filled in, they will be marked as missing items and assigned a priority based on the importance of the field. At the same time, conditional required items in specific nursing scenarios will also be checked, such as intravenous infusion records must contain information such as drug name, dosage, rate, etc.; pain records must contain information such as pain score, location, nature, etc. Then, a logical consistency check is performed to verify whether the logical relationship between the various parts of the information in the record is reasonable. This includes temporal logic checks to ensure the chronological order of events is reasonable, such as medication administration should precede the time of drug reaction observation. Numerical logic checks to verify the relationship between relevant numerical values ​​is reasonable, such as the total volume of infusion should equal the sum of the doses of each component. State logic checks to verify the consistency of patient status descriptions, such as describing a patient as both "conscious" and "comatose." Causal logic checks to verify the causal relationship between treatment measures and outcomes is reasonable, such as ensuring pain assessments are available after analgesic administration. Medical plausibility checks are also performed to verify the medical plausibility of record content based on medical knowledge. Rules and constraints in the medical knowledge base are used to check whether the medical content in the records conforms to medical common sense and clinical standards. This includes medication plausibility checks to verify that drug dosage, route of administration, and frequency of administration are within reasonable ranges. Symptom and sign plausibility checks to verify that symptom descriptions and physical sign measurements support each other, such as describing a "fever" with a corresponding increase in body temperature. Treatment response plausibility checks to verify that the relationship between treatment measures and patient responses is consistent with medical expectations. Time series checks are also performed to analyze the plausibility of temporal trends in continuous records. Compare records of the same patient at different time points to check whether the changing trends of key indicators are in line with clinical expectations, such as whether changes in vital signs such as body temperature and blood pressure are within a reasonable range, and whether the treatment effect changes accordingly over time. Numerical range checks are also performed to verify whether various measurement values ​​are within normal or acceptable ranges. Use the medical reference range database to check whether various measurement values ​​such as body temperature, blood pressure, pulse, blood sugar, etc. are within the normal range, or whether the degree of deviation from the normal range is reasonable. Values ​​outside the normal range will be marked as potential abnormalities and assigned a warning level based on the degree of deviation. Based on the results of the above checks, intelligent completion suggestions are generated to help caregivers improve records.These suggestions include: missing item prompts, which indicate necessary information missing from the record and suggest additions; abnormal value warnings, which identify possible abnormal measurements and suggest rechecking or confirmation; logical error prompts, which point out logical contradictions or inconsistencies in the record and suggest corrections; and reference value suggestions, which provide reference values ​​or examples for certain fields to help caregivers fill in standard content. The generated intelligent completion suggestions not only identify problems but also provide specific improvement plans, allowing caregivers to quickly complete the record. These suggestions are based on medical knowledge and clinical standards, are professional and practical, and can effectively improve the quality of nursing records.

[0169] The specific implementation of medical common sense verification rules is based on the combination of rule engines and knowledge bases. Multiple types of verification rules are predefined: (1) Numerical range rules: define the normal range and warning thresholds of each vital sign, such as the normal range of adult body temperature is 36.3-37.2℃, and the warning threshold is <35.5℃ or >38.5℃; the normal range of blood pressure is 90-140mmHg systolic pressure and 60-90mmHg diastolic pressure, and the warning threshold is <80mmHg or >180mmHg systolic pressure and <50mmHg or >110mmHg diastolic pressure. (2) Time series rules: define the reasonable range of changes in the time series of measured values, such as the body temperature change within 1 hour usually does not exceed 1℃, and the blood pressure change within 1 hour under stable conditions usually does not exceed 20mmHg systolic pressure or 15mmHg diastolic pressure. (3) Drug dosage rules: Based on the drug knowledge base, the reasonable dosage range, dosing interval and contraindications of various drugs are defined. For example, the conventional oral dose of amoxicillin for adults is 250-500 mg, once every 8 hours; it is contraindicated for patients allergic to penicillin. (4) Logical consistency rules: Define the logical constraints in clinical descriptions. For example, when consciousness is described as "awake", the descriptions of "coma" or "unconscious" should not appear at the same time. The forward-linking reasoning engine is used to perform rule verification. For any content found to be inconsistent with the rules, a warning level (general, important, urgent) is assigned according to the degree of deviation, and targeted correction suggestions are generated. The rule base supports dynamic updates through configuration files and can be adjusted according to the hospital's specific standards and the latest clinical guidelines.

[0170] Step S1065: presenting the intelligent completion suggestion to the nursing staff and optimizing the initially formatted nursing record after obtaining confirmation to obtain the standardized nursing record.

[0171] In this step, the intelligent completion suggestions generated in step S1064 are presented to the caregiver, and the final optimization of the nursing record is achieved through human-computer interaction, resulting in a high-quality, standardized nursing record. This step embodies the principle of human-computer collaboration, leveraging the algorithm's efficiency and accuracy while retaining the judgment and decision-making power of professionals, ensuring the professionalism and reliability of the final record. First, intelligent completion suggestions are presented through an intuitive user interface, allowing caregivers to clearly understand potential issues and areas for improvement in the record. Suggestions are categorized by type (e.g., missing items, outliers, logical errors) and severity (e.g., urgent, important, general), helping caregivers prioritize important issues. Each suggestion includes a description of the problem, a cause analysis, and improvement suggestions, enabling caregivers to fully understand the issue and make professional judgments. Intuitive visual markers, such as different colors and icons, are used to identify different types and severities of issues, improving readability. An interactive suggestion processing interface then provides support for caregivers to address each suggestion. Caregivers can choose to accept, modify, or reject the suggestion, and can add justifications or additional explanations. For accepted suggestions, corresponding modifications are automatically applied, such as filling in missing items, correcting outliers, and adjusting logical errors. For suggestions that the caregiver decides to modify, convenient editing tools are provided, allowing for in-context modification of relevant content. For suggestions rejected by the caregiver, the reason for rejection is recorded for subsequent algorithm optimization and learning. Batch processing is also supported, allowing caregivers to handle multiple suggestions of the same type in a unified manner, improving operational efficiency. After the caregiver has processed all suggestions, the pre-formatted nursing record is optimized based on the results to generate the final standardized nursing record. This optimization includes: content supplementation (filling missing information based on accepted suggestions); value correction (correcting or marking special cases based on identified outliers); logical adjustment (adjusting the presentation or order of relevant content based on logical errors); and format standardization (ensuring that the final record format complies with the hospital's electronic medical record and nursing record standards). The record's metadata is also updated, recording information such as the last modification time, modification person, and modification content to ensure record traceability. The resulting standardized nursing record is not only accurate and complete in content, but also standardized and formatted to meet the hospital's electronic medical record requirements. A record review function is also provided, allowing the head nurse or physician to review and make necessary corrections to automatically generated records. Reviewers can add comments, modify record content, or return records for rewriting. Records that pass review are marked as "reviewed" and officially stored in the electronic medical record. Feedback from nursing staff during use, including the acceptance rate of suggestions, modifications made, and reasons for rejection, is collected to continuously improve and optimize the record generation algorithm.Through this human-computer collaboration, high-quality nursing records can be generated that meet regulatory requirements and reflect actual nursing situations, significantly improving the efficiency and quality of nursing records.

[0172] Please refer to Figure 3 , Figure 3 This is a structural block diagram of the intelligent nursing record generation device provided by an embodiment of the present invention. Figure 3 As shown, the device includes:

[0173] The voice data preprocessing module 801 is used to obtain nursing voice data, perform noise reduction and sound enhancement processing on the nursing voice data, and obtain a preprocessed nursing voice data stream;

[0174] A speech recognition module 802 is configured to recognize medical terminology based on the pre-processed nursing speech data stream using speech recognition technology that integrates ECAPA-TDNN and x-vectors to obtain text transcription results;

[0175] A target speaker recognition module 803 is configured to identify the speech of a target nurse by combining a speaker-independent model and a speaker-conditional model with the preprocessed nursing speech data stream and the text transcription result to obtain a target speaker speech transcription text;

[0176] The speech analysis and structured processing module 804 is used to perform speech analysis and structured processing based on the target speaker's speech transcription text using a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework to obtain structured text with clinical semantic features;

[0177] A clinical information extraction module 805 is configured to extract key clinical information from nursing records based on the structured text with clinical semantic features and in combination with a medical ontology knowledge base using a deep learning key information extraction algorithm to obtain a standardized set of nursing record elements;

[0178] The nursing record generation module 806 is used to automatically generate a nursing record document that complies with nursing standards based on the standardized nursing record element set through a template generation algorithm to obtain a standardized nursing record.

[0179] An embodiment of the present invention may also provide a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned intelligent nursing record generation method when executed by a processor.

[0180] An embodiment of the present invention may also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned intelligent nursing record generation method when executing the computer program.

[0181] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for generating intelligent nursing records, characterized in that: include: Acquiring nursing voice data, performing noise reduction and sound enhancement processing on the nursing voice data, and obtaining a pre-processed nursing voice data stream; Based on the pre-processed nursing voice data stream, medical professional terminology is recognized using speech recognition technology that integrates ECAPA-TDNN and x-vector to obtain text transcription results; Recognizing the speech of the target nursing staff by combining a speaker-independent model and a speaker-conditional model with the preprocessed nursing speech data stream and the text transcription result to obtain a target speaker speech transcription text; Based on the target speaker's speech transcription text, a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework are used to perform speech analysis and structural processing to obtain structured text with clinical semantic features; Based on the structured text with clinical semantic features, combined with the medical ontology knowledge base, a deep learning key information extraction algorithm is used to extract key clinical information from the nursing record to obtain a standardized nursing record element set; Based on the standardized nursing record element set, a nursing record document that complies with nursing standards is automatically generated through a template generation algorithm to obtain a standardized nursing record.

2. The method according to claim 1, characterized in that The performing noise reduction and sound enhancement processing on the nursing voice data to obtain a pre-processed nursing voice data stream includes: Using an adaptive noise cancellation algorithm to perform real-time identification and filtering of medical scenario-specific noise in the nursing voice data to obtain preliminary noise-reduced voice data; Based on the preliminarily denoised speech data, spectral subtraction and multi-microphone beamforming technology are applied to enhance the human voice frequency band signal and suppress environmental reverberation, thereby obtaining acoustically enhanced high-quality speech data; The acoustically enhanced high-quality voice data is segmented into breathing blocks using Silero voice activity detection technology and timestamps and scene identifiers are added to obtain the pre-processed nursing voice data stream.

3. The method according to claim 1, characterized in that The speech recognition technology integrating ECAPA-TDNN and x-vector is used to identify medical terminology and obtain text transcription results, including: Based on the preprocessed nursing speech data stream, an ECAPA-TDNN network is used to extract speech features and a speech recognition model optimized for medical terminology is constructed to obtain a preliminary speech feature representation; receiving the preliminary speech feature representation, integrating ECAPA-TDNN and x-vector speaker representation technology to establish a decoupled representation of speaker identity and speech content through a contrastive learning method, thereby obtaining an enhanced speech feature with speaker information; Based on the enhanced speech features with speaker information, an attention mechanism Transformer model optimized for a medical terminology library is applied to perform speech-to-text conversion to obtain the text transcription result.

4. The method according to claim 1, wherein The method of recognizing the speech of the target caregiver by combining the technology of the speaker-independent model and the speaker-conditional model to obtain the target speaker speech transcription text includes: Using a speaker-independent model, a binary classifier is trained on the preprocessed nursing speech data stream and the text transcription result to automatically detect multi-speaker scenes in the speech and obtain time markers of speech overlapping areas; The x-vector technology is used to extract the voiceprint features of each speaker and match them with the pre-registered nursing staff voiceprint database to obtain the speaker identification result; Based on the speaker identification result and the time mark of the speech overlapping area, the feature linear modulation technology is used to integrate the speaker embedding and the speaker conditional model is applied to process the overlapping speech part to obtain the target speaker speech transcription text.

5. The method according to claim 1, wherein The multimodal self-supervised learning model and the hierarchical CNN-BiLSTM framework are used to perform speech analysis and structural processing to obtain structured text with clinical semantic features, including: Using pre-trained Wav2Vec2, HuBERT, and WavLM self-supervised learning models to extract acoustic and semantic features of the target speaker's speech transcription text from different dimensions to obtain a multimodal feature representation; Dividing the multimodal feature representation into breathing chunks based on Silero voice activity detection technology and analyzing chunk-level fluency metrics including speech rate, pause duration, and repetition pattern to obtain prosodic speech features; The outputs of different self-supervised learning models are fused through a learnable weighting mechanism, combined with the prosodic speech features, to balance the contributions of acoustic and linguistic features to obtain a comprehensive feature representation; A hierarchical CNN-BiLSTM network architecture is applied to capture local and long-term dependencies and perform deep semantic analysis on the comprehensive feature representation to obtain the structured text with clinical semantic features.

6. The method according to claim 1, characterized in that The medical ontology knowledge base is combined with a deep learning key information extraction algorithm to extract key clinical information from nursing records, and a standardized nursing record element set is obtained, including: Based on clinical nursing standards and professional guidelines, a multi-dimensional medical ontology knowledge base including nursing operations, drugs, symptoms, and signs is constructed to obtain a structured medical ontology knowledge model; Using the structured medical ontology knowledge model, performing entity recognition and relationship extraction on the structured text with clinical semantic features, mapping the descriptive language in the structured text with clinical semantic features to standard medical terminology to obtain a standardized clinical terminology set; Applying a deep learning key information extraction algorithm to automatically identify and rate important clinical information in the standardized clinical terminology set to obtain a clinical information set with importance tags; Classifying and arranging the clinical information set with importance marks into element groups that conform to the nursing record structure according to the element model of the nursing record standard to obtain the standardized nursing record element set; The medical ontology knowledge model is continuously learned and updated, and the structure of the medical ontology knowledge base is optimized according to newly added clinical cases and nursing records to obtain a dynamically updated medical ontology knowledge model.

7. The method according to claim 1, characterized in that The template generation algorithm automatically generates nursing record documents that comply with nursing standards to obtain standardized nursing records, including: Based on the hospital's existing nursing record specifications and electronic medical record standards, a template library covering different nursing scenarios was constructed to obtain a dynamic nursing record template set; According to the content characteristics and context information of the standardized nursing record element set, a template matching algorithm is applied to automatically select the most suitable nursing record template to obtain a matching record template; Intelligently filling the clinical information in the standardized nursing record element set according to the matched record template and adding metadata including a timestamp and an operation type to obtain a preliminarily formatted nursing record; Using medical common sense verification rules and completeness check algorithms to identify missing items and logical errors in the initially formatted nursing records, and obtain intelligent completion suggestions; After confirmation by the nursing staff, the initially formatted nursing record is optimized according to the intelligent completion suggestion to obtain the standardized nursing record.

8. The method according to claim 6, characterized in that The deep learning key information extraction algorithm is applied to automatically identify and rate the important clinical information in the standardized clinical terminology set to obtain a clinical information set with importance marks, including: Constructing a bidirectional long short-term memory network model based on an attention mechanism to semantically encode clinical terms in the standardized clinical term set to obtain a semantic vector representation of the term; Using a conditional random field model to perform sequence annotation on the semantic vector representation of the term to identify key clinical entity types and obtain entity annotation results; Applying a graph convolutional neural network to analyze the dependencies and semantic associations between entities in the entity annotation results to obtain an entity relationship graph; An importance scoring model is constructed based on medical expert knowledge and clinical guidelines to quantitatively evaluate the importance of the clinical information in the entity relationship diagram to obtain an importance score; Priority marks and urgency marks are added to the clinical information in the entity relationship diagram according to the importance score to obtain the clinical information set with importance marks.

9. The method according to claim 4, characterized in that The method of integrating speaker embedding using the feature linear modulation technique and applying the speaker conditional model to process overlapping speech parts to obtain the target speaker speech transcription text includes: Inputting the speaker embedding vector in the speaker identification result into a feature linear modulation network, dynamically modulating the speech features by learning scaling and offset parameters to obtain speaker conditional features; Constructing a speaker-conditional model and training it on the synthesized mixed speech data so that the speaker-conditional model focuses on the speech content of the target speaker, thereby obtaining a trained speaker-conditional model; Using the trained speaker conditional model, the overlapping speech segments corresponding to the time markers of the speech overlapping regions are processed to obtain separated speech content of the target nursing staff; Establish a dynamic speaker tracking mechanism to associate the voice clips of the same nurse in different time periods through continuous matching of voiceprint features to obtain the associated voice clips; The associated speech segments are integrated into a complete narrative based on time sequence and semantic coherence while retaining timestamp information to obtain a speech transcription text of the target speaker.

10. An intelligent nursing record generating device, characterized in that: include: A voice data preprocessing module is used to obtain nursing voice data, perform noise reduction and sound enhancement processing on the nursing voice data, and obtain a preprocessed nursing voice data stream; A speech recognition module is used to identify medical professional terms based on the preprocessed nursing speech data stream using speech recognition technology that integrates ECAPA-TDNN and x-vector to obtain text transcription results; a target speaker recognition module for identifying the speech of a target caregiver by combining a speaker-independent model and a speaker-conditional model with the preprocessed nursing speech data stream and the text transcription result to obtain a target speaker speech transcription text; A speech analysis and structured processing module is used to transcribe the target speaker's speech into text, perform speech analysis and structured processing using a multimodal self-supervised learning model and a hierarchical CNN-BiLSTM framework, and obtain structured text with clinical semantic features; A clinical information extraction module is used to extract key clinical information from nursing records based on the structured text with clinical semantic features and in combination with a medical ontology knowledge base through a deep learning key information extraction algorithm to obtain a standardized set of nursing record elements; The nursing record generation module is used to automatically generate a nursing record document that complies with nursing standards based on the standardized nursing record element set through a template generation algorithm to obtain a standardized nursing record.

Images