Outpatient medical record generation method and system based on end side model

Abstract

The invention discloses an outpatient medical record generation method and system based on an end-side model, and relates to the technical field of smart medical treatment.The method comprises the steps that a voice collection module is started, and doctor-patient conversation audios are collected through a directional microphone array; the main control module performs voice recognition on the acquired doctor-patient conversation audio by using a low-power-consumption NPU chip and an end side model carried by the main control module, converts the doctor-patient conversation audio into text information, performs role separation on the doctor-patient conversation audio through the end side model, and distinguishes the conversation contents of a doctor and a patient; and the main control module analyzes and understands the text information after role identification and separation by using the end side model. The system has a voice acquisition function, a voice recognition function and a role separation function, the recognized voice content is analyzed, understood and standardized by deploying an end side model, and the outpatient electronic medical record is automatically generated in combination with electronic medical record writing specifications and a local medical knowledge base. And the content is transmitted to a doctor workstation through the communication module to realize electronic medical record write-back.

Description

Technical Field

[0001] The present invention relates to the field of smart medical technology, and in particular to a method and system for generating outpatient medical records based on a terminal-side model. Background Art

[0002] Because hospitals are uniquely situated and cannot connect to the external network, the mainstream approach is to deploy a private large-scale model on a server within the hospital's intranet. Microphones are installed at the doctor's office to capture audio of doctor-patient conversations and transmit it to the server, which then performs speech recognition. The large-scale model then summarizes the information and composes the document. Based on basic electronic medical record writing standards, the outpatient electronic medical record is automatically generated. The current deployment method is costly and cumbersome, hindering widespread adoption. Running the large-scale model locally requires GPU-powered servers, and the voice acquisition terminal only captures audio, resulting in a waste of resources. Summary of the Invention

[0003] The purpose of the present invention is to provide an outpatient medical record generation method and system based on an end-side model to solve the problems in the background technology.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for generating outpatient medical records based on a client-side model, comprising the following steps: Start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; The main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, converting the audio into text information. The on-device model then separates the roles in the audio and distinguishes the content of the doctor-patient conversation. The main control module uses the client-side model to analyze and understand the text information after character identification and separation, extracting key information. Combining the local medical knowledge base and medical record templates in the storage module, the main control module standardizes key information and automatically generates outpatient electronic medical records in accordance with electronic medical record writing standards; The generated outpatient electronic medical records are transmitted to the doctor's workstation through the communication module.

[0005] In a preferred embodiment, the step of starting the voice acquisition module and using the directional microphone array to collect the audio of the doctor-patient conversation includes: Set the hardware parameters for the microphone array; The digital signal is transmitted to the voice acquisition module through the data bus, and the voice acquisition module synchronously processes the multi-channel digital audio signal.

[0006] The audio signal is analyzed and divided into speech signal and noise signal. The statistical characteristics of the noise signal are used to construct an adaptive filter. The parameters are adjusted in real time according to the noise changes to eliminate the noise components and obtain the audio of the doctor-patient conversation.

[0007] In a preferred embodiment, the main control module uses its low-power NPU chip and device-side model to perform speech recognition on the collected audio of the doctor-patient conversation, convert the audio into text information, and use the device-side model to separate the roles of the doctor-patient conversation audio and distinguish the content of the doctor-patient conversation. The steps include: The convolutional neural network is trained using labeled data to obtain an on-device model. Use an on-device model to recognize speech in audio conversations between doctors and patients; Use the GMM-UBM algorithm to collect doctors' speech samples and build a general background model; During real-time voice collection, the audio is framed and processed, and the voiceprint features of each frame are extracted and matched with the doctor's GMM model to determine whether each frame of voice belongs to the doctor in real time. Real-time voice that does not belong to the doctor is classified as patient voice, thus completing the separation of doctor and patient conversation content.

[0008] In a preferred embodiment, the main control module uses the client-side model to analyze and understand the text information after role identification and separation, and extract key information, including: The text information of the doctor and patient conversations is converted into structured representations, and the text information is quantified through feature word selection and weight assignment; Represent the text and image elements in the text information as nodes in the original image; Extract key document information from the original image, use a CNN backbone network to extract high-level visual information, use an entity category-aware attention network to learn entity layout information, and use an LSTM-based decoder to decode the preset entity content as a document element; Encode the extracted document element image, strengthen the connection of the internal information of the image through the attention module, and finally obtain the information recognition of the document element through the decoder; Graph neural networks are used to model document elements and extract key information from documents. MLP is used to determine the information category of document elements and evaluate the correlation between elements as the basis for key information.

[0009] In a preferred embodiment, the steps of combining the local medical knowledge base and medical record template in the storage module, standardizing the key information by the main control module, and automatically generating an outpatient electronic medical record in accordance with the electronic medical record writing specification include: Search the local medical knowledge base based on key information to determine where the key information belongs in the medical knowledge system; Standardize the medical terms in key information according to unified medical standards; According to the patient's condition type and the department of treatment, select the appropriate medical record template from the medical record template library in the storage module.

[0010] In a preferred embodiment, the step of transmitting the generated outpatient electronic medical record to the doctor's workstation via the communication module includes: Convert the generated outpatient electronic medical records from internal storage format to a standard format suitable for network transmission; Encapsulate the outpatient electronic medical records and encrypt the encapsulated data using symmetric or asymmetric encryption algorithms; The encrypted outpatient electronic medical records are transmitted according to the network protocol until they are transmitted to the doctor's workstation.

[0011] The present invention also provides an outpatient medical record generation system based on a terminal-side model, comprising: The acquisition module is used to start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; The separation module is connected to the acquisition module. The main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, convert the audio into text information, and use the on-device model to separate the roles in the audio of doctor-patient conversations and distinguish the conversation content between doctors and patients. The analysis module is connected to the separation module and is used by the main control module to analyze and understand the text information after role identification and separation using the client-side model to extract key information; The generation module is connected to the analysis module and is used to combine the local medical knowledge base and medical record template in the storage module. The main control module standardizes the key information and automatically generates outpatient electronic medical records according to the electronic medical record writing specifications; The transmission module is connected to the generation module and is used to transmit the generated outpatient electronic medical records to the doctor's workstation through the communication module.

[0012] In the above technical solution, the technical effects and advantages provided by the present invention are: This system features voice acquisition, voice recognition, and role separation. By deploying a client-side model, it analyzes, understands, and standardizes the recognized voice content. Combining this with electronic medical record writing standards and the local medical knowledge base, it automatically generates outpatient electronic medical records. This content is then transmitted to the doctor's workstation via a communication module, enabling electronic medical record writeback. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] Figure 1 Flow chart of the method of the present invention.

[0015] Figure 2 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0017] Example 1, please refer to Figure 1 As shown, the outpatient medical record generation method based on the end-side model described in this embodiment includes the following steps: S1. Start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; S11, setting hardware parameters for the microphone array; S12. Transmit the digital signal to the voice acquisition module via the data bus, and the voice acquisition module synchronously processes the multi-channel digital audio signal.

[0018] S13. Analyze the audio signal and divide it into speech signal and noise signal. Use the statistical characteristics of the noise signal to build an adaptive filter. Adjust the parameters in real time according to the noise changes to eliminate the noise component and obtain the audio of the doctor-patient conversation.

[0019] During the acquisition process, the collected audio is processed with the help of noise suppression, sound source localization, voice separation functions, and array dereverberation algorithms to ensure that the audio quality meets the requirements of subsequent processing; It's important to note that the voice acquisition module initializes: When the outpatient medical record generation terminal is started, the module first performs initialization. This involves setting the hardware parameters of the directional microphone array, such as adjusting the microphone's sensitivity and gain, to ensure it accurately captures the audio signals of the doctor-patient conversation. Simultaneously, the module performs a self-test of the signal processing circuitry within the module, checking the proper functioning of various components, such as the audio amplifier and analog-to-digital converter (ADC). This ensures that the audio signal is smoothly converted from analog to digital, preparing for subsequent processing. Connection status confirmation: The voice acquisition module also needs to establish a stable communication connection with the main control module. Using a specific communication protocol (such as I2C or SPI), the module sends a connection request to the main control module and waits for its response. Once the connection is established, the module transmits its status information (such as the microphone array's operating status and acquisition frequency) to the main control module, allowing the main control module to monitor the voice acquisition module's operation in real time.

[0020] Working principle of a directional microphone array: A directional microphone array consists of multiple microphones arranged in a certain geometric shape. Common arrangements include linear arrays and circular arrays. When collecting audio, each microphone will receive sound signals from different directions. Due to the different propagation paths and attenuation levels of sound in different directions, the signals received by each microphone will differ in amplitude, phase, and other aspects. By analyzing and processing these differences, functions such as sound source localization and noise suppression can be achieved. Multi-channel audio acquisition: The directional microphone array used in this system has 4-6 channels, with one microphone for each channel. During the acquisition process, each microphone independently converts the received analog sound signal into a digital signal and transmits the digital signal to the voice acquisition module via the data bus. The module will synchronously process these multi-channel digital audio signals to ensure that the data of each channel is aligned in time for subsequent joint processing.

[0021] Adaptive Noise Cancellation Algorithm: An adaptive noise cancellation algorithm is used to suppress ambient noise. This algorithm analyzes the audio signal collected by the microphone and separates it into two components: speech and noise. It leverages the statistical properties of the noise signal to construct an adaptive filter that adjusts its parameters in real time based on noise variations, effectively eliminating the noise component from the mixed signal. For example, in an outpatient setting, there may be noise such as the sound of air conditioning running or other patients talking. The adaptive noise cancellation algorithm dynamically tracks the characteristics of these noises and removes them from the audio of the doctor-patient conversation.

[0022] Spectral subtraction: Spectral subtraction is also a commonly used noise suppression method. It first performs a Fourier transform on the audio signal, converting it from the time domain to the frequency domain. Then, by estimating the spectral characteristics of the noise, the noise spectrum is subtracted from the spectrum of the mixed signal to obtain the spectrum of the pure speech. Finally, an inverse Fourier transform is performed to convert the frequency domain signal back to the time domain. Spectral subtraction can effectively reduce the noise amplitude, but it may introduce side effects such as musical noise to a certain extent, requiring optimization in combination with other methods.

[0023] Localization methods based on time difference of arrival (TDOA) use the time difference between sound signals received by different microphones in a microphone array to localize a sound source. As sound propagates from the source to each microphone, the time it reaches each microphone varies due to the different distances it travels. By measuring these time differences and combining them with the geometric position information of the microphone array, the direction and distance of the sound source can be calculated. For example, suppose a sound propagates from a certain direction to the microphone array, and the time difference between the sound signals received by microphones 1 and 2 is Δt. Given the speed of sound in air as v and the distance between microphones 1 and 2 as d, the angle of the sound source relative to the microphone array can be calculated based on the geometric relationship. Beamforming technology: Beamforming forms a beam pointed in a specific direction by weighted summing the signals received by the microphone array, thereby enhancing the sound signal from that direction and suppressing noise and interference from other directions. During sound source localization, other methods (such as TDOA) can be used to roughly determine the direction of the sound source. The beamformer's weights can then be adjusted to align the beam in that direction, further improving the signal-to-noise ratio of the sound signal.

[0024] Deep Learning Model: A deep learning-based voice separation model, such as one combining a convolutional neural network (CNN) and a recurrent neural network (RNN), is used. This model first extracts features from multi-channel audio signals, such as time-frequency features and Mel-Frequency Cepstral Coefficients (MFCCs). These features are then fed into the model for training and prediction. The model learns the characteristic differences between human voices and non-human voices (such as noise and other sounds), thereby separating human voices from mixed signals. During training, supervised learning using a large amount of audio data containing both human and non-human voices is required to improve the model's separation performance. Independent Component Analysis (ICA): ICA is a statistical signal processing method that assumes that a mixed signal is a linear combination of multiple independent source signals. By applying ICA to the mixed signal, these independent source signals can be separated. In audio processing of doctor-patient conversations, ICA can treat human voices and noise as independent source signals, thereby achieving voice separation. However, when processing audio signals in complex environments, ICA may be affected by factors such as nonlinear signal mixing, and the separation effect may not be as good as that of deep learning models.

[0025] Inverse filtering algorithm: The inverse filtering algorithm estimates the impulse response of the room, constructs an inverse filter, and convolves it with the audio signal collected by the microphone to eliminate the effects of reverberation. The impulse response of the room describes the signal changes caused by factors such as reflection and scattering in the room during the process of sound propagating from the sound source to the microphone. By measuring the impulse response of the room, an inverse filter can be constructed so that the signal after inverse filtering is as close as possible to the original direct sound signal. De-reverberation method based on deep learning: In recent years, deep learning has also made significant progress in the field of dereverberation. Deep learning models such as convolutional neural networks (CNN) or generative adversarial networks (GAN) can be used to learn the mapping relationship between reverberation signals and clean signals. By training on a large amount of data containing reverberation and clean audio, the model can automatically learn the characteristics and rules of dereverberation, thereby achieving effective processing of reverberant audio. This method does not require precise estimation of the impulse response of the room and has stronger adaptability and robustness; The S2 main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, converting the audio into text information. The on-device model then separates the roles in the audio of doctor-patient conversations and distinguishes the content of the conversation between the doctor and the patient. S21. Train the convolutional neural network using the labeled data to obtain an on-device model. S22: Use the on-device model to perform speech recognition on the audio of the doctor-patient conversation. S23. Use the GMM-UBM algorithm to collect the doctor's voice samples and build a general background model; S24. When collecting real-time voice, the audio is divided into frames and processed to extract the voiceprint features of each frame. The features are matched with the doctor's GMM model to determine whether each frame of voice belongs to the doctor in real time. The real-time voice that does not belong to the doctor is classified as the patient's voice, thus completing the separation of the doctor and patient's conversation content.

[0026] It should be noted that the end-to-end speech recognition model uses the Conformer model. The Conformer model combines the advantages of convolutional neural networks (CNNs) and transformers. The CNN component effectively extracts local features, while the transformer excels at capturing long-range dependencies. This combination results in higher accuracy and robustness in speech recognition tasks. Support for mixed Chinese and English input: To accommodate mixed Chinese and English expressions (such as drug names and professional terminology) that may occur in outpatient clinics, the model is trained using a dataset containing a large amount of mixed Chinese and English corpus. This corpus covers common medical terms, drug names, and symptom descriptions, ensuring the model can accurately recognize mixed Chinese and English speech content. Guaranteed recognition accuracy: By continuously optimizing the model structure and training parameters, and using large-scale, high-quality annotated data for training, the model achieves a recognition accuracy of over 95%. Furthermore, after the model is deployed, voice data from real-world applications is regularly collected for model updates and optimization to further improve recognition accuracy.

[0027] Voiceprint feature extraction: Voiceprint feature extraction is performed using the GMM-UBM (Gaussian Mixture Model-Universal Background Model) algorithm. First, voice samples from doctors and patients are collected to construct a universal background model (UBM). The UBM is a model that describes the statistical characteristics of a large number of voice samples and is derived through cluster analysis of large amounts of voice data. The UBM is then adaptively adjusted using the doctor's and patient's voice samples, resulting in separate GMM models for each doctor and patient. These GMM models accurately describe the voiceprint characteristics of both doctors and patients.

[0028] Conversation Content Separation: During real-time voice collection, the collected audio is framed and the voiceprint features of each frame are extracted. The extracted voiceprint features are matched with the GMM models of the doctor and patient, and the matching results determine whether each frame belongs to the doctor or the patient. This method separates the content of the doctor-patient conversation, providing an accurate source of information for subsequent medical record generation. Model Selection: Deploy a lightweight large model combining MobileBERT and a rule engine. MobileBERT is a lightweight version of the BERT model. Using techniques such as knowledge distillation, it significantly reduces the number of model parameters and computational complexity while maintaining high performance, making it suitable for running on device-side devices. The rule engine is used to process specific medical rules and logic, such as the correlation between symptoms and diagnoses, and the usage and dosage of medications. Integrating ICD-10 diagnostic codes with a local drug library: During medical record generation, the model generates structured medical records based on the recognized and separated voice content, combined with ICD-10 diagnostic codes and a local drug library. ICD-10 diagnostic codes are an internationally recognized disease classification system that accurately describes patient diagnoses. The local drug library contains information on commonly used medications in the hospital, such as drug names, specifications, and usage and dosage. Based on the patient's symptoms and the physician's diagnosis, the model selects appropriate medications from the local drug library and enters the diagnosis and medication information into the medical record in a standardized format. Voice Command Recognition: To facilitate real-time adjustments to generated medical record content, the system supports dynamic revisions via voice commands. Physicians can modify generated medical record content by speaking specific voice commands, such as "Change the chief complaint to headache" or "Add the diagnosis to hypertension." The system uses voice recognition technology to convert the physician's voice commands into text and perform semantic interpretation. Command Parsing and Execution: After receiving the physician's voice command, the system parses it to determine its specific meaning and action target. Based on the parsed results, the generated medical record content is modified accordingly. For example, if the physician's command is "Change the chief complaint to headache," the system locates the chief complaint section in the medical record and modifies it to "headache." Once the modification is complete, the system regenerates a structured medical record and displays it on the physician's workstation for further confirmation. Speech Acquisition: The speech acquisition module uses a directional microphone array to capture audio of doctor-patient conversations. It processes the audio using noise suppression, sound source localization, voice separation, and an array dereverberation algorithm to improve audio quality. Speech Recognition and Character Separation: The processed audio is input into the Conformer end-to-end speech recognition model for mixed Chinese and English speech recognition, generating text information. Simultaneously, the GMM-UBM algorithm is used to extract voiceprint features and separate the conversation content between the doctor and patient. Information Analysis and Comprehension: The separated text information is analyzed and understood to extract key information, such as symptoms, medical history, and diagnostic strategies.Medical record generation: The extracted information is fed into the lightweight, large-scale model of the MobileBERT+ rules engine, combined with ICD-10 diagnostic codes and the local drug library to generate structured medical records. Dynamic revisions: Doctors can make real-time revisions to generated medical records using voice commands. The system then interprets and executes the commands, regenerating the records. S3: The main control module uses the on-device model to analyze and understand the text information after role identification and separation, extracting key information. S31, converting the text information of the conversation between the doctor and the patient into a structured representation, and quantifying the text information by selecting feature words and assigning weights; S32, representing the text and image elements in the text information as nodes in the original image; S33, extracting key document information from the original image, using a CNN backbone network to extract high-level visual information, using an entity category-aware attention network to learn entity layout information, and using an LSTM-based decoder to decode and obtain the preset entity content as a document element; S34, encoding the extracted document element image, strengthening the connection of internal information of the image through the attention module, and finally obtaining information recognition of the document element through the decoder; S35. Use graph neural networks to model document elements and extract key information from the document; use MLP to determine the information category of document elements and evaluate the correlation between elements as key information.

[0029] It should be noted that the vector space model converts unstructured text into structured representation and quantifies text information through feature word selection and weight assignment. Feature words must meet the requirements of content recognition, differentiation, quantity controllability, and ease of implementation. Feature word selection method: Mapping transformation method: map the original features into fewer features.

[0030] Representative selection method: Select representative features from the original features.

[0031] Expert evaluation method: select the most influential features based on expert experience.

[0032] Mathematical model method: Use mathematical models to find the features with the greatest classification ability.

[0033] Raster-based method: Convert document images into raster representation vectors and input them into a deep learning network to extract key information.

[0034] Graph-based method: The text and image elements in the document are represented as nodes in the graph, and the relationships between them are represented as edges in the graph. The document information is understood and analyzed by constructing a graph structure.

[0035] An end-to-end approach considers the entire document image processing process as a whole, completing all tasks through a single network model. The EATEN method extracts key document information directly from the raw image input. It uses a CNN backbone network to extract high-level visual information, an entity-aware attention network to learn entity layout information, and finally uses an LSTM-based decoder to decode the pre-determined entity content. This method better captures and processes document context and layout information, achieving high accuracy and robustness. It is suitable for recognizing and processing complex document images, such as invoices and contracts.

[0036] Visual Detection Module (Layout Detection): Detects various elements in a document, including text, stamps, headers, footers, tables, and more. It uses downsampling, upsampling, and attention modules to obtain multi-scale, spatial, and channel-related features of the image, and uses a point-wise approach to obtain heatmap areas for document elements.

[0037] Character recognition module (OCR): Encodes the extracted document element image, strengthens the connection of internal information in the image through the attention module, and finally obtains information recognition of the document element through the decoder.

[0038] Semantic Coordination Module (NLP): Utilizes semantic models to reason about the semantic rationality of information identified by OCR, adjusts information content, and improves information recognition accuracy. Based on proprietary NLP information extraction technology, it can intelligently analyze and extract key information from single-page or multi-page documents of any format without the need for configuration or training.

[0039] Information Extraction Module (GNN): Utilizes graph neural networks to model layout elements and extract key information from documents. Multimodal information, including text and images (document element ROIs), is fed into the GNN (graph neural network model). Image information is encoded as edges within the GNN, representing relationships between elements; text information is encoded as nodes within the GNN, representing element information. Finally, an MLP (Multi-Layer Perceptron) is used to determine the information category of the elements and assess inter-element relevance. S4. Combining the local medical knowledge base and medical record template in the storage module, the main control module standardizes the key information and automatically generates an outpatient electronic medical record in accordance with the electronic medical record writing specifications; S41. Search the local medical knowledge base based on the key information to determine where the key information belongs in the medical knowledge system; S42. Standardize the medical terms in key information according to unified medical standards; S43. Select a suitable medical record template from the medical record template library in the storage module according to the patient's condition type and the department where the patient is visiting.

[0040] It should be noted that, key information acquisition and preprocessing: Acquisition of key information: The main control module receives processed text information from the speech recognition and role separation module. This information already contains key content such as symptoms, medical history, and diagnostic ideas. Information cleaning: The acquired text information is cleaned to remove irrelevant characters, repeated content, incorrect expressions, etc. to ensure the accuracy and consistency of the information. For example, unnecessary pause words and repeated medical terms that may be generated during the speech recognition process are removed. Information classification: According to the knowledge system in the medical field, the cleaned key information is classified, such as symptom information is classified into one category, medical history information is classified into one category, diagnostic idea information is classified into one category, etc., to facilitate subsequent standardized processing.

[0041] Retrieval and matching of local medical knowledge base: The main control module accesses the local medical knowledge base in the storage module, which contains a wealth of medical knowledge, such as disease symptoms, diagnostic criteria, treatment methods, drug information, etc. Based on the classified key information, the knowledge base is searched to find relevant medical knowledge. For example, if the key information contains the symptom of "headache", disease information related to headache is retrieved in the knowledge base. Information matching: The retrieved medical knowledge is matched with the key information to determine the exact meaning and attribution of the key information in the medical knowledge system. For example, through matching, it is determined that "headache" is a specific symptom of a certain disease, and the range of diseases that this symptom may correspond to.

[0042] Standardization of key information: Medical terms in key information are standardized according to unified medical standards to ensure the accuracy and consistency of the terms. For example, the colloquial expression "headache" is standardized as "headache". Format standardization: The format of key information is standardized to meet the requirements of electronic medical record writing standards. For example, the date format is unified as "YYYY-MM-DD" and the time format is unified as "HH:MM:SS". Logical relationship sorting: The logical relationship between key information is sorted out, such as the causal relationship between symptoms and diseases, the correlation between medical history and current condition, etc., so as to accurately express them in the medical record. For example, clarify the potential connection between the symptom of "headache" and the patient's previous history of hypertension.

[0043] Template selection: Select a suitable medical record template from the medical record template library in the storage module based on the patient's condition and the department they are visiting. The medical record template library contains many different types of medical record templates, such as internal medicine medical record templates, surgical medical record templates, pediatric medical record templates, etc. Each template is designed in accordance with the electronic medical record writing specifications. Information filling: Fill the standardized key information into the selected medical record template. According to the format and requirements of the template, fill in the symptoms, medical history, diagnostic ideas and other information in the corresponding positions. For example, fill in the patient's symptom information in the "chief complaint" section, fill in the medical history information in the "current medical history" and "past medical history" sections, and fill in the diagnostic ideas information in the "diagnosis" section.

[0044] Medical Record Review and Correction: The main control module automatically reviews completed medical records to check for compliance with electronic medical record standards, such as missing information, logical errors, or non-standard formatting. For example, it checks whether the necessary personal information is included and whether the diagnostic evidence is sufficient. If the automatic review identifies any issues, the main control module provides feedback to the doctor, who then makes corrections to the record based on the feedback. Doctors can supplement, modify, or delete key information to ensure the accuracy and completeness of the record.

[0045] Medical record generation: After review and revision, the main control module generates the final outpatient electronic medical record in accordance with the requirements of the electronic medical record writing specifications. The generated medical record contains detailed information such as the patient's personal information, symptoms, medical history, diagnosis, treatment plan, etc., with standardized format and accurate content. Medical record storage: The generated outpatient electronic medical record is stored in the hospital's information system so that doctors can review and access it at any time. At the same time, the medical record is stored in the storage module and managed and maintained as the hospital's medical data assets; S5. Transmit the generated outpatient electronic medical record to the doctor's workstation through the communication module (realizing electronic medical record writing back for further review, modification and improvement by the doctor); S51, converting the generated outpatient electronic medical record from an internal storage format to a standard format suitable for network transmission; S52. Encapsulate the outpatient electronic medical record and encrypt the encapsulated data using a symmetric encryption or asymmetric encryption algorithm; S53, transmitting the encrypted outpatient electronic medical record according to the network protocol until it is transmitted to the doctor's workstation; It should be noted that medical record data encapsulation is the process of converting the generated outpatient electronic medical records from an internal storage format (such as database records, specific document formats, etc.) into a standard format suitable for network transmission, most commonly XML or JSON formats. For example, if a medical record contains basic patient information (name, age, gender), symptom description, diagnosis results, treatment plan, etc., this information is organized according to the structural rules of XML or JSON. Taking XML as an example, different tags can be defined to represent different information fields, such as <patientname>Indicates the patient's name, <symptoms>Indicates symptom description, etc.

[0046] Data Encryption: To ensure the security of medical records during transmission, encapsulated data is encrypted using either symmetric or asymmetric encryption algorithms. Symmetric encryption algorithms, such as AES (Advanced Encryption Standard), use the same key for both encryption and decryption; asymmetric encryption algorithms, such as RSA, use a public key for encryption and a private key for decryption. For example, when using AES encryption, a random encryption key is generated, the medical records are encrypted with this key, and then the encryption key is encrypted with the doctor's public key. This allows only the doctor with the corresponding private key to decrypt and access the medical records.

[0047] Communication protocol selection and configuration: Select a suitable communication protocol based on the hospital's network environment and security requirements. Common protocols include HTTP / HTTPS protocol, FTP protocol, etc. The HTTPS protocol adds an SSL / TLS encryption layer on the basis of HTTP, which can provide more secure communication protection and is suitable for transmitting sensitive medical records data. For example, when the hospital's internal network environment is good and the security requirements are high, the HTTPS protocol is preferred. Protocol configuration: Configure the selected communication protocol, including setting the server address, port number, authentication information, etc. If the HTTPS protocol is used, the server's SSL certificate needs to be configured to ensure the identity authentication of both communicating parties. For example, when configuring the HTTPS server, install the hospital's digital certificate on the server and set the certificate's private key path, password and other information.

[0048] Target address determination: Doctor workstations typically have unique identifiers, such as IP addresses, domain names, or specific device numbers. The main control module needs to obtain the identification information of the target doctor workstation. For example, the IP address of the doctor workstation can be queried through the hospital's internal management system. This IP address uniquely identifies the workstation's location within the network. Address verification: The obtained target address is verified to ensure its correctness and reachability. Simple test packets (such as the ping command) can be sent to check whether the target address is responding properly. For example, a ping request can be sent to the doctor workstation's IP address. If a reply is received, the address is reachable.

[0049] Data Transmission: Based on the selected communication protocol, the main control module establishes a network connection with the doctor's workstation. If HTTPS is used, the main control module, acting as the client, initiates a connection request to the doctor's workstation (server). After an SSL / TLS handshake, both parties negotiate encryption algorithms, keys, and other parameters, establishing a secure communication channel. The encapsulated and encrypted medical record data is then sent to the doctor's workstation via the established connection. During transmission, data can be divided into blocks based on a specific packet size to improve efficiency and reliability. For example, medical record data can be divided into multiple 1024-byte packets and sent sequentially to the destination. During data transmission, the main control module monitors the transmission status in real time, including transmission speed, progress, and errors. If any transmission errors, such as packet loss or network interruption, are detected, retransmission or other error handling actions are performed promptly. For example, if a packet is detected to have been unsuccessfully sent, the main control module automatically resends it.

[0050] Receiving and decrypting the data at the doctor's workstation: The doctor's workstation listens to a designated port. When it receives data from the main control module, it stores the data in a temporary buffer. The workstation needs to continuously check the buffer to ensure complete data reception. For example, the workstation can set a timer to periodically check the buffer for new data. The doctor's workstation decrypts the received encrypted data using the corresponding private key (for asymmetric encryption) or encryption key (for symmetric encryption). After successful decryption, the data is restored to the original outpatient electronic medical record format. For example, the encryption key is decrypted using the RSA private key, and the decrypted AES key is then used to decrypt the medical record data.

[0051] Example 2, please refer to Figure 2 As shown, the outpatient medical record generation system based on the end-side model described in this embodiment includes: The acquisition module is used to start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; The separation module is connected to the acquisition module. The main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, convert the audio into text information, and use the on-device model to separate the roles in the audio of doctor-patient conversations and distinguish the conversation content between doctors and patients. The analysis module is connected to the separation module and is used by the main control module to analyze and understand the text information after role identification and separation using the client-side model to extract key information; The generation module is connected to the analysis module and is used to combine the local medical knowledge base and medical record template in the storage module. The main control module standardizes the key information and automatically generates outpatient electronic medical records according to the electronic medical record writing specifications; The transmission module is connected to the generation module and is used to transmit the generated outpatient electronic medical records to the doctor's workstation through the communication module.

[0052] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.< / symptoms> < / patientname>

Claims

1. The outpatient medical record generation method based on the end-side model is characterized by: The following steps are involved: Start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; The main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, converting the audio into text information. The on-device model then separates the roles in the audio and distinguishes the content of the doctor-patient conversation. The main control module uses the client-side model to analyze and understand the text information after character identification and separation, extracting key information. Combining the local medical knowledge base and medical record templates in the storage module, the main control module standardizes key information and automatically generates outpatient electronic medical records in accordance with electronic medical record writing standards; The generated outpatient electronic medical records are transmitted to the doctor's workstation through the communication module.

2. The outpatient medical record generation method based on the end-side model according to claim 1 is characterized in that: The step of starting the voice acquisition module and using the directional microphone array to collect the audio of the doctor-patient conversation includes: Set the hardware parameters for the microphone array; The digital signal is transmitted to the voice acquisition module through the data bus, and the voice acquisition module synchronously processes the multi-channel digital audio signal.

3. Analyze the audio signal and divide it into speech signal and noise signal. Use the statistical characteristics of the noise signal to build an adaptive filter. Adjust the parameters in real time according to the noise changes to eliminate the noise component and obtain the audio of the doctor-patient conversation.

4. The outpatient medical record generation method based on the end-side model according to claim 1 is characterized in that: The main control module uses its low-power NPU chip and device-side model to perform speech recognition on the collected audio of the doctor-patient conversation, convert the audio into text information, and use the device-side model to separate the roles in the audio of the doctor-patient conversation and distinguish the conversation content between the doctor and the patient. The steps include: The convolutional neural network is trained using labeled data to obtain an on-device model. Use an on-device model to recognize speech in audio conversations between doctors and patients; Use the GMM-UBM algorithm to collect doctors' speech samples and build a general background model; During real-time voice collection, the audio is framed and processed, and the voiceprint features of each frame are extracted and matched with the doctor's GMM model to determine whether each frame of voice belongs to the doctor in real time. Real-time voice that does not belong to the doctor is classified as patient voice, thus completing the separation of doctor and patient conversation content.

5. The outpatient medical record generation method based on the end-side model according to claim 1 is characterized in that: The main control module uses the client-side model to analyze and understand the text information after role identification and separation, and extract key information, including the following steps: The text information of the doctor and patient conversations is converted into structured representations, and the text information is quantified through feature word selection and weight assignment; Represent the text and image elements in the text information as nodes in the original image; Extract key document information from the original image, use a CNN backbone network to extract high-level visual information, use an entity category-aware attention network to learn entity layout information, and use an LSTM-based decoder to decode the preset entity content as a document element; Encode the extracted document element image, strengthen the connection of the internal information of the image through the attention module, and finally obtain the information recognition of the document element through the decoder; Graph neural networks are used to model document elements and extract key information from documents. MLP is used to determine the information category of document elements and evaluate the correlation between elements as key information.

6. The outpatient medical record generation method based on the end-side model according to claim 1 is characterized in that: The main control module standardizes key information by combining the local medical knowledge base and medical record template in the storage module and automatically generates outpatient electronic medical records according to the electronic medical record writing specifications, including: Search the local medical knowledge base based on key information to determine where the key information belongs in the medical knowledge system; Standardize the medical terms in key information according to unified medical standards; According to the patient's condition type and the department of treatment, select the appropriate medical record template from the medical record template library in the storage module.

7. The outpatient medical record generation method based on the end-side model according to claim 1 is characterized in that: The step of transmitting the generated outpatient electronic medical record to the doctor's workstation via the communication module includes: Convert the generated outpatient electronic medical records from internal storage format to a standard format suitable for network transmission; Encapsulate the outpatient electronic medical records and encrypt the encapsulated data using symmetric or asymmetric encryption algorithms; The encrypted outpatient electronic medical records are transmitted according to the network protocol until they are transmitted to the doctor's workstation.

8. An outpatient medical record generation system based on a device-side model, used to implement the outpatient medical record generation method based on a device-side model according to any one of claims 1 to 6, characterized in that: include: The acquisition module is used to start the voice acquisition module and use the directional microphone array to collect the audio of the doctor-patient conversation; The separation module is connected to the acquisition module. The main control module uses its low-power NPU chip and on-device model to perform speech recognition on the collected audio of doctor-patient conversations, convert the audio into text information, and use the on-device model to separate the roles in the audio of doctor-patient conversations and distinguish the conversation content between doctors and patients. The analysis module is connected to the separation module and is used by the main control module to analyze and understand the text information after role identification and separation using the client-side model to extract key information; The generation module is connected to the analysis module and is used to combine the local medical knowledge base and medical record template in the storage module. The main control module standardizes the key information and automatically generates outpatient electronic medical records according to the electronic medical record writing specifications; The transmission module is connected to the generation module and is used to transmit the generated outpatient electronic medical records to the doctor's workstation through the communication module.

Images