Medical record robot based on AI voice chip SMCU and Chinese language model

Abstract

A medical record robot based on an AI voice chip SMCU and a Chinese language model comprises a voice collector used for collecting doctor-patient dialogue data, the AI voice chip SMCU, an edge calculation box integrated with the Chinese language model and a data center, and the AI voice chip SMCU is connected with the voice collector and the edge calculation box; the voice collector, the AI voice chip SMCU, the edge computing box and the interaction device are all connected with the power supply module. The Chinese language model comprises MedSegNet, Input Embedding, Nx, Bert-BiLSTM-CRF (Conditional Random Access Function), a GCN (Graphics Convolutional Network) and GraphRAG (Graphics Register Architecture). The system can automatically record conversations between doctors and patients, extract and analyze structured information in the conversations, generate electronic medical records in real time, query past medical histories of the patients recorded in the system, and give corresponding treatment opinions for the doctors to refer to and modify. The busy work of writing medical records by doctors is greatly liberated, and the diagnosis and treatment efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical device technology, and in particular relates to a medical record robot based on an AI voice chip SMCU and a Chinese language model, Background Art

[0002] A medical record robot is a device that integrates artificial intelligence technology. It can automatically record doctor-patient conversations and generate electronic medical records through voice recognition and natural language processing capabilities. This robot can significantly improve the efficiency of medical record writing, reduce the paperwork burden of doctors, and ensure the accuracy and structure of medical records. The design concept of the medical record robot is to reduce the labor time of doctors writing medical records, transfer more time to communication with patients, improve the personalization of the diagnosis and treatment process and the accuracy of the diagnosis and treatment results. It usually has the following functions:

[0003] 1. Sound signal collection and analysis function: collect and record the sound signals of the conversation between doctors and patients, extract the effective signals, and convert them into text information.

[0004] 2. Structured information extraction: The conversation between doctors and patients contains invalid or unstructured information. The robot needs to have an in-depth and accurate understanding of the conversation content and convert it into structured medical terms.

[0005] 3. Semantic understanding and electronic medical record generation functions: The robot needs to have an in-depth understanding of these structured information and combine it with its own expert knowledge base to generate electronic medical records that meet the standards.

[0006] 4. Privacy data storage function: The patient's medical consultation information involves privacy and needs to be stored securely to provide reference information for subsequent diagnosis and treatment.

[0007] Among the above functions, structured information extraction and electronic medical record generation are the core functions of medical record robots, which can assist doctors in completing a large number of medical record writing. However, in existing application scenarios, doctors have limited outpatient time every day, and the processing methods of converting sound signals into text based on mathematical models or deep learning methods have high latency, which will eventually waste the time of many doctors and patients, resulting in tight and inefficient consultation work. Secondly, the patient's geographical background is relatively complex, and differences in verbal expression ability and dialect differences will affect the extraction of structured information, and ultimately affect the accuracy of medical record writing. In addition, it is also necessary to conduct a comprehensive and accurate pre-assessment of the patient's condition based on the generation of structured medical records and generate a reference diagnosis and treatment plan to reduce the workload of doctors and improve the efficiency of diagnosis and treatment. Summary of the invention

[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a medical record robot based on an AI voice chip SMCU and a language model that can automatically record conversations between doctors and patients, extract and analyze structured information therein, and generate electronic medical records in real time.

[0009] The object of the present invention is achieved through the following technical solutions: A medical record robot based on an AI voice chip SMCU and a Chinese language model, comprising:

[0010] Voice collector: used to collect the conversation data between doctors and patients, convert the collected data into digital signals, and send them to the AI ​​voice chip SMCU;

[0011] AI voice chip SMCU: performs noise reduction, silence removal, pre-emphasis, and frame windowing operations on the digital signal sent by the voice collector to optimize its signal quality; then uses the pyannote algorithm to separate the mixed voices of multiple people to obtain an output tag sequence; then the AI ​​voice chip extracts features from the sequence, inputs the extracted features into the neural network processor NPU, and outputs the text encoding information of the human natural language corresponding to the voice signal, which will be input into the subsequent language model;

[0012] Edge computing box: It integrates Chinese language models, performs structured extraction and semantic analysis on input text encoding information, generates electronic medical records that meet national standards, and can calculate auxiliary diagnosis opinions based on the patient's previous medical history;

[0013] The language model includes the medical segmentation network MedSegNet, the input embedding module Input Embedding, the Nx module, the Bert-BiLSTM-CRF, the graph convolutional network GCN, and the graph retrieval enhancement generation module GraphRAG.

[0014] (1) MedSegNet consists of three modules: a context-aware module that adjusts the segmentation boundaries based on the context, a task-aware module that adjusts the segmentation granularity according to task requirements, and a medical dictionary-assisted module that uses medical dictionaries to identify and preserve the integrity of medical terms.

[0015] The goal of the context-aware module is to use context information to dynamically determine word boundaries, use a BiLSTM network to extract context information, and use context information to determine whether each character should be used as a word boundary;

[0016] The task-aware module adjusts the word segmentation granularity according to the type of downstream tasks. It works in the following ways: it influences the word segmentation strategy through a set task identifier and passes task category information to the word segmentation model;

[0017] The medical dictionary auxiliary module is used to identify and prioritize the terms in medical texts. It contains a glossary of common diseases, drug names, and symptoms. When performing word segmentation, the current character sequence is first matched with the dictionary. If the current character sequence belongs to a medical term, it is directly marked as a complete word without being split. At the same time, the terms in the dictionary have a higher word segmentation priority.

[0018] The word segmentation is achieved by obtaining the word segmentation boundary through the context perception module, and then the word segmentation result is adjusted according to the task perception module and the medical dictionary auxiliary module, and finally the word segmentation sequence is output;

[0019] (2) Input Embedding maps the word sequence after word segmentation into a vector representation to capture the semantic features of the words;

[0020] (3) The Nx module adopts the standard Transformer framework, which includes an input layer, an encoder layer, and an output layer. The encoder layer can capture the deep semantic relationships and contextual dependencies in medical texts by stacking multiple layers of self-attention mechanisms.

[0021] (4) Bert-BiLSTM-CRF is used to identify medical entities in text. In the BERT-BiLSTM-CRF model, the input text is first encoded through the BERT layer as follows:

[0022] H BERT =BERT(X′)

[0023] H BERT is the hidden state sequence feature output by the BERT layer, and X′ is the word embedding sequence output by the Nx module;

[0024] After the BERT layer, a bidirectional long short-term memory network BiLSTM is used to further extract features;

[0025] Finally, conditional random fields (CRF) are used to output medical entities in the text.

[0026] (5) The nodes input to GCN are the named entities output by Bert-BiLSTM-CRF, and then Spacy is used to construct dependency relationships, that is, the initialization edges between nodes; finally, the GCN algorithm is used to learn complex relationship expressions and obtain an entity relationship graph; through the entity relationships generated by GCN, information is retrieved from the Medical KG knowledge graph to generate standard electronic medical records;

[0027] (6) GraphRAG retrieves information by integrating the Medical KG and medical documents to obtain auxiliary diagnosis opinions;

[0028] The data center is used to store all recorded data of doctor-patient conversations, as well as the data generated in the middle and electronic medical records;

[0029] The AI ​​voice chip SMCU is connected to the voice collector and the edge computing box respectively; the voice collector, the AI ​​voice chip SMCU and the edge computing box are all connected to the power module.

[0030] The beneficial effects of the present invention are as follows: This application uses existing advanced sensor and smart chip technologies, as well as effective deep learning methods, to address the problem that most of the doctor's energy is transferred to medical record writing during the actual consultation process, and the communication with the patient is not in-depth and careful enough, and proposes an intelligent device to solve this problem. During each patient's communication with the doctor, the doctor will quickly set some guiding questions for the medical record robot to avoid the patient's inability to express himself clearly and to help the doctor quickly locate the patient's symptoms. After that, the medical record robot will use acoustic sensors and sound collectors to record and store the conversation between the patient and the doctor in real time. The conversation record will be transmitted to a data center connected to the intranet for storage. At the same time, the AI ​​voice chip SMCU analyzes and processes the collected acoustic signals, and its built-in voice recognition algorithm performs noise reduction, text translation and other operations. After that, the information processed by the AI ​​voice chip SMCU is transmitted to the edge The edge computing box has a built-in semantic model, namely the Chinese language model. It will determine which specialist field the patient's condition belongs to, and hand it over to the expert system of its model to perform complex understanding and calculation on the information. Finally, it will output the structured electronic medical record in real time and display it on the monitor in front of the doctor. After the consultation, the edge computing box will also retrieve the patient's medical history from the data center through the intranet and give auxiliary treatment suggestions. The electronic medical record and treatment suggestions can be modified by the doctor twice to a small extent to correct the existing deviations. Finally, the doctor's electronic signature is confirmed, and the medical record officially takes effect and enters the data center. In this process, doctors hardly need to write medical records manually, and only need to make small modifications in the final stage. Most of the time, doctors are communicating with patients about their conditions. Doctors can more accurately grasp the condition and give the best treatment plan, which also improves the level of humanistic care. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a structural schematic diagram of the medical record robot based on the AI ​​voice chip SMCU and the Chinese language model of the present invention;

[0032] Figure 2 It is a structural diagram of the language model in the present invention. DETAILED DESCRIPTION

[0033] In order to make the technical problems solved by the present invention, the technical solutions adopted and the effects achieved clearer, the technical solutions of the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0034] In the description of the present invention, unless otherwise clearly specified and limited, the terms "connected", "connected", and "fixed" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0036] like Figure 1 As shown, a medical record robot based on an AI voice chip SMCU and a language model of the present invention includes:

[0037] Voice collector: used to collect the conversation data between doctors and patients, and convert the collected data into digital signals, and send them to the AI ​​voice chip SMCU. The voice collector captures sound signals through the built-in sound sensor, converts them into electrical signals, and samples them at a discrete sampling rate of not less than 48kHz, converts them into digital signals using pulse code modulation (PCM), and encapsulates the digital signals into frames of a certain length and sends them to the AI ​​voice chip SMCU. A number of acoustic sensors are arranged around the voice collector, and are arranged in an array near the sound collector to sensitively capture the acoustic signals of doctor-patient conversations; the acoustic sensors are electrically connected to the sound collector.

[0038] The AI ​​voice chip SMCU is a system microcontroller unit for voice recognition and text conversion. It integrates AMD's Ryzen AI 300NPU, with a system frequency of up to 411MHz, built-in SRAM up to 2MB, and integrates multiple micro-units such as RC oscillator and PMU power management unit, supporting multi-model parallel processing and joint decision-making. The main parameters of the AI ​​voice chip SMCU are as follows:

[0039]

[0040]

[0041] AI voice chip SMCU: It performs noise reduction, silence removal, pre-emphasis, and frame windowing operations on the digital signal sent by the voice collector to optimize its signal quality; then uses the pyannote algorithm to separate the mixed voices of multiple people and represents the input digital signal as a digital signal sequence of indefinite length. pyannote gives a tag sequence to the sequence As a prediction, the solution process is expressed as a conditional probability distribution P(Y (1) ,Y (2) ,…,Y (n) |X (1) ,X (2) ,…,X (n) ), the labeling system finds the corresponding output tag sequence for the new input observation sequence according to the learned conditional probability distribution model, that is, finds the tag sequence y that maximizes the conditional probability N+1 The features to be extracted;

[0042] The AI ​​voice chip then extracts features from the sequence after the above processing, and inputs the extracted features into the neural network processor NPU. In the NPU, firmware is used instead of software to implement neural network calculations. The NPU uses the recurrent neural network RNN, Whisper and WaveNet to complete the calculation and conversion of digital signals in parallel. Finally, the outputs of the three are combined and the average is taken as the final voice conversion result. The text encoding information of the human natural language corresponding to the voice signal is output, and the encoding information will be input into the subsequent language model.

[0043] Edge computing box: It integrates Chinese language models, performs structured extraction and semantic analysis on input text encoding information, generates electronic medical records that meet national standards, and can calculate auxiliary diagnosis opinions based on the patient's previous medical history;

[0044] The edge computing box can be freely configured to meet the reasoning computing power of the Chinese language model, that is, it includes CPU, memory, hard disk, GPU, communication module, etc., which is similar to a microcomputer, but in particular, it has a built-in Chinese language model and has the function of interacting with patients and doctors, and its interactive function is realized by the interactive module intermediary communication; Chinese language model, in this embodiment, the Chinese language model is a deep learning model, a model for medical semantic analysis, which can extract structured information of doctor-patient dialogues, build knowledge graphs, generate electronic medical records, and generate auxiliary diagnosis and treatment opinions; interactive module, in this embodiment, the interactive module mainly includes a touch screen for patients and a high-color depth display for doctors. The touch screen is placed in front of the patient's consultation seat. The patient can click with his fingers or the touch screen. The touch screen can display some necessary information to the patient. The doctor can set up a medical consultation questionnaire that meets the characteristics of the department in advance and prompt the user to answer to help the doctor quickly locate the patient's condition. The high-color depth display is mainly used by doctors, and the doctor can view the patient's basic information on it. Information and real-time generated medical records information. At the same time, for departments such as orthopedics that need to view medical images, high-color-depth displays have better display effects. At the same time, some operations performed by doctors on high-color-depth displays will be saved by the edge computing box and synchronously updated to the data center. In fact, the high-color-depth display can also be realized through the data interface led out by the edge computing box, and the actual display uses the hospital's existing display equipment, which can reduce costs; power module. In this embodiment, the power module provides power for several acoustic sensors, touch screens, high-color-depth displays, edge computing boxes, and voice collectors. The implementation details are to introduce a hub and electrically connect the above-mentioned multiple devices to the rich types of interfaces on the hub; data center. In this embodiment, the data center stores all information about the patient, divided according to different information sources, including the original acoustic signals of the doctor-patient dialogue, the doctor-patient dialogue information processed by the AI ​​voice chip SMCU, the electronic medical records generated by the edge computing box, the valid electronic medical records modified and confirmed by the doctor, and the doctor's operation information about the edge computing box.

[0045] The specific structure of the language model is as follows Figure 2 As shown, it includes the medical segmentation network MedSegNet, the input embedding module Input Embedding, the Nx module, the Bert-BiLSTM-CRF, the graph convolutional network GCN, and the graph retrieval enhancement generation module GraphRAG;

[0046] (1) MedSegNet, the full name of which is Medical Segmentation Network, is designed to improve word segmentation accuracy. It has a dynamic adaptive word segmentation mechanism that can automatically adjust the word segmentation strategy according to the context of the input text, task requirements, and requirements of specific medical fields. It is used to segment the input text encoding; it includes three modules: a context-aware module that adjusts the word segmentation boundary based on the context, a task-aware module that adjusts the word segmentation granularity according to task requirements, and a medical dictionary auxiliary module that uses medical dictionaries to identify and retain the integrity of medical terms;

[0047] The goal of the context-aware module is to use context information to dynamically determine word segmentation boundaries. This part relies on modeling the context of each character in the text and using a BiLSTM network to extract context information. It receives text information output from the AI ​​smart voice chip SMCU and performs a multi-step classification on each character x input. i Extract context information and generate context vector representation Follow the following algorithm:

[0048]

[0049] Contextual information is used to determine whether each character should be used as a word boundary (for example, whether it is the beginning or end of a word). If a character has a strong semantic connection in the context (such as the context before and after a drug name), its word boundary may need to be adjusted.

[0050] Task-aware word segmentation granularity In order to dynamically adjust the word segmentation granularity according to different tasks, a task-aware mechanism is required. The core of this mechanism is implemented through the task-aware module, which adjusts the word segmentation granularity according to the type of downstream tasks. For example, for disease prediction tasks, larger semantic units (such as disease names) may need to be processed as a whole, while for drug naming and recognition tasks, each drug name may need to be identified as an independent entity. The task-aware module works in the following ways: through a set task identifier (such as disease prediction, drug recognition, etc.) to influence the word segmentation strategy, by passing task category information to the word segmentation model; adding task label input to the model so that the model can adjust the word segmentation granularity according to task requirements during the reasoning process. For example, in the drug naming task, "amoxicillin" may be required to be processed as a whole word, while in the disease prediction task, "gastric cancer" may be fine-grained. The algorithm follows the following formula:

[0051]

[0052] Among them, Granularity is the granularity of word segmentation, which depends on the task type (such as disease classification, drug identification, etc.).

[0053] The medical dictionary auxiliary module is used to identify and prioritize the terms in medical texts to avoid incorrect segmentation; it contains a glossary of common diseases, drug names, and symptoms; by matching the terms in these dictionaries, the model can ensure that these terms are not split incorrectly. When performing word segmentation, the current character sequence is first matched to the dictionary; if the current character sequence belongs to a medical term (such as the drug name "amoxicillin"), it is directly marked as a complete word without being split; at the same time, the terms in the dictionary have a higher word segmentation priority; even if the BiLSTM model recommends splitting the current sequence, the term priority in the dictionary will override these suggestions. The algorithm flow is as follows:

[0054]

[0055] The word segmentation is achieved by obtaining the word segmentation boundary through the context perception module, and then the word segmentation result is adjusted according to the task perception module and the medical dictionary auxiliary module, and finally the word sequence after word segmentation is output.

[0056] (2) Input Embedding maps the word sequence after word segmentation into a vector representation to capture the semantic features of the words; the expression is as follows:

[0057] Embedding = Word Embedding + Position Embedding + Segment Embedding WordEmbedding, Position Embedding and Segment Embedding are the pre-trained vectors of vocabulary, the vector representation of each position and different text fragments respectively; after adding the embedding vectors of all word segments, they are input into the Nx module;

[0058] (3) The Nx module adopts the standard Transformer framework, which includes an input layer, an encoder layer, and an output layer. The encoder layer can capture the deep semantic relationships and contextual dependencies in medical texts by stacking multiple layers of self-attention mechanisms.

[0059] The calculation process of attention is as follows:

[0060]

[0061] Input embedding inputs the embedding vector X to the input layer of the Nx module. The embedding vector is transformed into Q, K, and V through three different linear transformations; specifically, for each element in the input sequence, there are:

[0062] Q=XWQ ,K=XW K ,V=XW V

[0063] W Q , W K , and W V is the weight matrix corresponding to query, key, and value; in the multi-head attention mechanism, Q, K, and V will be divided into multiple heads, each head calculates attention separately, and then the results of all heads are concatenated and linearly transformed, and then the processed word embedding sequence X′ is obtained through the output layer;

[0064] (4) Bert-BiLSTM-CRF is used to identify medical entities in text. In the BERT-BiLSTM-CRF model, the input text is first encoded through the BERT layer as follows:

[0065] H BERT =BERT(X′)

[0066] H BERT It is the hidden state sequence feature output by the BERT layer;

[0067] After the BERT layer, a bidirectional long short-term memory network BiLSTM is used to further extract features;

[0068] Finally, the conditional random field CRF is used to output the medical entities in the text. The CRF layer uses the Viterbi algorithm to find the most likely label sequence; the output of CRF is expressed as:

[0069]

[0070] Among them, Y* is the most likely label sequence; P(yi|y i -1,h i ) is given the previous label y i -1 and hidden state h i Under the condition that the current label y i The conditional probability of; in CRF, the conditional probability is calculated by the following formula:

[0071]

[0072] score(y i ,y i -1,h i ) is a scoring function, expressed as:

[0073]

[0074] (5) GCN is a graph convolutional network. The input nodes are the named entities output by Bert-BiLSTM-CRF. Spacy is then used to construct dependency relationships, i.e., the initialization edges between nodes. Finally, the GCN algorithm is used to learn complex relationship expressions and obtain an entity relationship graph. The entity relationships generated by GCN are used to retrieve information from the Medical KG knowledge graph to generate standard electronic medical records. The retrieval uses the following rule matching algorithm:

[0075]

[0076] Context(w i , ±n) is based on w i centered, the context fragment with window size n, S, They are entity relationship graph and medical knowledge graph Medical KG respectively; finally, the output standard electronic medical record is constructed;

[0077] (6) The graph retrieval enhancement generation module GraphRAG (Graph-based Reinforcement Augmented Generation) retrieves information by integrating the knowledge graph Medical KG and medical documents to obtain auxiliary diagnosis opinions. GraphRAG uses reinforcement learning (RL) as the main control strategy to optimize the data flow and decision-making process in the graph, especially in medical decision support systems. This is achieved through a reward mechanism based on the effectiveness of the segmentation, diagnosis or treatment strategy generated by the system based on the doctor-patient interaction data. Among them, each state s t is a characteristic representation of a node in the graph, such as a disease node or a symptom node. Each action a t is the decision made based on the current state, such as adding a new connection to node i, or adjusting the node label (for example, changing the relationship between symptoms and diseases). t , a t ) is used to evaluate the effectiveness of each decision. For example, a high reward is given for the correct matching of drugs and diseases. The reinforcement learning optimization formula is as follows:

[0078]

[0079] Where α is the learning rate, γ is the discount rate, and a′ is the next action.

[0080] The data center is used to store all recorded data of doctor-patient conversations, as well as the data generated in the middle and electronic medical records;

[0081] The AI ​​voice chip SMCU is connected to the voice collector and the edge computing box respectively; the voice collector, the AI ​​voice chip SMCU, the edge computing box, and the interactive device are all connected to the power module.

[0082] The interactive device is electrically connected to the edge computing box, and the source data sent from the edge computing box changed by the interactive device is stored in the edge device again;

[0083] The data center is connected to the edge computing box and uses intranet communication. The data center stores all the data of the edge computing box. The data center communicates with the equipment involved in the present invention in an intranet manner. The data center is deployed on the data server of the hospital, which avoids the risk of data leakage caused by networking.

[0084] Other peripherals can be added to the robot of the present invention as needed.

[0085] The touch screen is used to display diagnosis and treatment question and answer information to patients. Patients can provide feedback on their medical conditions to doctors through the touch screen. The touch screen is part of the interactive device and is connected to the edge computing box.

[0086] The high color depth display is used for doctors to view real-time generated electronic medical records and auxiliary diagnosis opinions. At the same time, doctors can view the patient's medical history and medical images, and can edit and feedback the calculation results of the edge computing box in real time; the high color depth display is part of the interactive device and is connected to the edge computing box for communication.

[0087] The power hub provides a variety of standard interfaces to ensure stable working power for acoustic sensors, sound collectors, edge computing boxes, touch screens and high color depth displays; the power hub is part of the power module.

[0088] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A medical record robot based on AI voice chip SMCU and Chinese language model, characterized by: include: Voice collector: used to collect the conversation data between doctors and patients, convert the collected data into digital signals, and send them to the AI ​​voice chip SMCU; AI voice chip SMCU: performs noise reduction, silence removal, pre-emphasis, and frame windowing operations on the digital signal sent by the voice collector to optimize its signal quality; then uses the pyannote algorithm to separate the mixed voices of multiple people to obtain an output tag sequence; then the AI ​​voice chip extracts features from the sequence, inputs the extracted features into the neural network processor NPU, and outputs the text encoding information of the human natural language corresponding to the voice signal, which will be input into the subsequent language model; Edge computing box: It integrates Chinese language models, performs structured extraction and semantic analysis on input text encoding information, generates electronic medical records that meet national standards, and can calculate auxiliary diagnosis opinions based on the patient's previous medical history; The language model includes the medical segmentation network MedSegNet, the input embedding module Input Embedding, the Nx module, the Bert-BiLSTM-CRF, the graph convolutional network GCN, and the graph retrieval enhancement generation module GraphRAG. (1) MedSegNet consists of three modules: a context-aware module that adjusts the segmentation boundaries based on the context, a task-aware module that adjusts the segmentation granularity according to task requirements, and a medical dictionary-assisted module that uses medical dictionaries to identify and preserve the integrity of medical terms. The goal of the context-aware module is to use context information to dynamically determine word boundaries, use a BiLSTM network to extract context information, and use context information to determine whether each character should be used as a word boundary; The task-aware module adjusts the word segmentation granularity according to the type of downstream tasks. It works in the following ways: it influences the word segmentation strategy through a set task identifier and passes task category information to the word segmentation model; The medical dictionary auxiliary module is used to identify and prioritize the terms in medical texts. It contains a glossary of common diseases, drug names, and symptoms. When performing word segmentation, the current character sequence is first matched with the dictionary. If the current character sequence belongs to a medical term, it is directly marked as a complete word without being split. At the same time, the terms in the dictionary have a higher word segmentation priority. The word segmentation is achieved by obtaining the word segmentation boundary through the context perception module, and then the word segmentation result is adjusted according to the task perception module and the medical dictionary auxiliary module, and finally the word segmentation sequence is output; (2) Input Embedding maps the word sequence after word segmentation into a vector representation to capture the semantic features of the words; (3) The Nx module adopts the standard Transformer framework, which includes an input layer, an encoder layer, and an output layer. The encoder layer can capture the deep semantic relationships and contextual dependencies in medical texts by stacking multiple layers of self-attention mechanisms. (4) Bert-BiLSTM-CRF is used to identify medical entities in text. In the BERT-BiLSTM-CRF model, the input text is first encoded through the BERT layer as follows: H BERT =BERT(X′) H BERT is the hidden state sequence feature output by the BERT layer, and X′ is the word embedding sequence output by the Nx module; After the BERT layer, a bidirectional long short-term memory network BiLSTM is used to further extract features; Finally, conditional random fields (CRF) are used to output medical entities in the text. (5) The nodes input to GCN are the named entities output by Bert-BiLSTM-CRF, and then Spacy is used to construct dependency relationships, that is, the initialization edges between nodes; finally, the GCN algorithm is used to learn complex relationship expressions and obtain an entity relationship graph; through the entity relationships generated by GCN, information is retrieved from the Medical KG knowledge graph to generate standard electronic medical records; (6) GraphRAG retrieves information by integrating the Medical KG and medical documents to obtain auxiliary diagnosis opinions; The data center is used to store all recorded data of doctor-patient conversations, as well as the data generated in the middle and electronic medical records; The AI ​​voice chip SMCU is respectively connected to the voice collector and the edge computing box; the voice collector, the AI ​​voice chip SMCU, and the edge computing box are all connected to the power module.

Images