An electronic medical record voice interaction device based on machine learning

Abstract

The present invention discloses an electronic medical record voice interaction device based on machine learning, including a voice input device, a client and a server, wherein the client is deployed with an electronic medical record system, and the server is deployed with a voice segmentation system, an input sampling system and a machine learning system; the voice input device collects voice information; the voice segmentation system converts the voice information into text information and processes the text information to generate an electronic medical record and then sends it to the electronic medical record system; the electronic medical record system displays the electronic medical record through the client and corrects the electronic medical record; the input sampling system collects the corrected electronic medical record and sends it to the voice segmentation system for processing and then provides it to the machine learning system for model training; the machine learning system provides an acoustic model, a language model and a writing model for the voice segmentation system. The present invention realizes accurate and efficient automatic generation of electronic medical records based on machine automatic learning, thereby improving the doctor's consultation efficiency and the accuracy of electronic medical records.

Description

Technical Field

[0001] The present invention relates to the field of electronic medical records, and in particular to an electronic medical record voice interaction device based on machine learning. Background Art

[0002] During the consultation process between doctors and patients, doctors prefer to communicate with patients face to face to provide better diagnosis and treatment services. Due to the development of electronic medical records and the requirements of hospital regulations, doctors need to face more and more complex electronic medical record entry, and the proportion of time facing the computer in the consultation time is gradually increasing. Less invasive and more convenient medical record entry methods are urgently needed to allow the consultation process to return to face-to-face communication.

[0003] With the maturity of speech recognition technology and machine learning technology, it has gradually become possible to use speech recognition technology with higher speech recognition rates and artificial intelligence that understands contextual reasoning better to help doctors communicate with patients and automatically write medical records at the same time.

[0004] The prior art patent CN107564571A uses a microphone device to allow doctors to input information into a structured medical record template by voice. The patent CN113724695A only uses the voice conversation between the doctor and the patient to determine the identity and automatically generate the medical record. Both use voice recognition technology to help doctors communicate with patients and automatically write medical records. However, there are the following defects:

[0005] 1) During this period, speech recognition was used to assist in data entry rather than assist in the consultation process. Doctors simply changed the way they read the medical records they wanted to enter word for word.

[0006] 2) Judging identity only through voice conversations between doctors and patients has a high misjudgment rate and cannot accurately deal with the common situation where a third party interrupts the conversation during the consultation. This is an important issue in realizing automatic voice entry into medical records during the consultation conversation.

[0007] 3) Doctors have accents and different vocabulary and abbreviation habits when speaking. The learning algorithm to improve the recognition rate is based on the correction results of all doctors. For each individual doctor, the voice input still has a high error rate;

[0008] 4) Although supervised learning can significantly improve the evolution of algorithms, it requires additional processes to prompt right and wrong answers, which is costly. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide an electronic medical record voice interaction device based on machine learning, which realizes accurate and efficient automatic generation of electronic medical records based on machine automatic learning.

[0010] The technical solution adopted by the present invention to solve the above-mentioned technical problems is to provide an electronic medical record voice interaction device based on machine learning, including a voice input device, a client and a server, the client is deployed with an electronic medical record system, and the server is deployed with a voice segmentation system, an input sampling system and a machine learning system; the voice input device collects voice information; the voice segmentation system converts the voice information into text information and processes the text information to generate an electronic medical record and then sends it to the electronic medical record system; the electronic medical record system displays the electronic medical record through the client and corrects the electronic medical record; the input sampling system collects the corrected electronic medical record and sends it to the voice segmentation system for processing, and then provides it to the machine learning system for model training; the machine learning system provides the voice segmentation system with an acoustic model, a language model and a writing model; the server is provided with a diagnosis and treatment voice database, a diagnosis and treatment text database, a dictionary, a medical terminology knowledge base, a dialect language base and a standard medical record template base.

[0011] Furthermore, the speech segmentation system includes a speech recognition module and an NLP module; the speech recognition module obtains speech information through a speech input device, and the speech information includes a medical consultation dialogue and an oral medical record; the speech recognition module obtains the identity information of both the doctor and the patient through an identity information acquisition device connected to the client; the speech recognition module converts the medical consultation dialogue voice information into a medical consultation dialogue text with identity information and time information according to the identity information, and the speech recognition module converts the oral medical record voice information into an oral medical record text with identity information and time information; the NLP module processes the medical consultation dialogue text and / or the oral medical record text to generate an electronic medical record and then sends it to the electronic medical record system.

[0012] Furthermore, the speech recognition module converts the speech information into sound vectors and then inputs them into the acoustic model for acoustic recognition and the language model for language recognition to generate a preliminary dialogue text; at the same time, the client monitors whether there is a change in identity information request through the connected identity information acquisition device, and the electronic medical record system monitors whether there is a switch in patient medical records, and determines whether there is a change in the dialogue characters based on the preliminary dialogue text, identifies and eliminates the dialogue content that is inserted in the preliminary dialogue text and is not related to the consultation, and generates a doctor-patient dialogue text with identity information and time information.

[0013] Furthermore, the diagnosis and treatment voice database stores historical diagnosis and treatment voice information, and the acoustic model is obtained by the machine learning system after obtaining historical diagnosis and treatment voice information from the diagnosis and treatment voice database, extracting voice features, and performing model training; the diagnosis and treatment text database stores historical doctor-patient conversation texts and oral medical record texts, and the language model is obtained by the machine learning system after obtaining historical doctor-patient conversation texts and oral medical record texts from the diagnosis and treatment text database, and performing model training; the dictionary connects the acoustic model and the voice model, and stores the correspondence between voice and text.

[0014] Furthermore, the NLP module performs word segmentation on the doctor-patient dialogue text to identify keywords in the doctor-patient dialogue text, converts the keywords into text in a daily writing style through a writing model, fills in the objective fact description part of the electronic medical record in combination with the electronic medical record writing requirements and sends it to the electronic medical record system; the objective fact description part of the electronic medical record includes the chief complaint part, the current medical history part and the past medical history part.

[0015] Furthermore, the NLP module preprocesses and segments the doctor-patient conversation text according to the diagnosis and treatment text database and the medical terminology knowledge base and obtains keywords; then, the medical named entity recognition is performed on the keywords through the rule set set by the rule description language; then, according to the entropy-expanded term extraction algorithm, it is determined whether the recognized keyword is a single keyword or a compound keyword; finally, the electronic medical record labels are extracted and the label vectors are constructed after filtering through the general vocabulary and removing the conventional keywords, and the various parts of the electronic medical record are filled according to the labels of the electronic medical record through the writing model.

[0016] Furthermore, the machine learning system includes an acoustic model learning module, a language model learning module and a writing model learning module; the medical terminology knowledge base, the dialect language library and the standard medical record template library are arranged in the machine learning system; the acoustic model learning module uses the voice information collected by the voice input device to generate a personalized acoustic model associated with the identity information; the language model learning module adjusts the language model by comparing the generated doctor-patient dialogue text with the revised electronic medical record; the writing model learning module adjusts the writing model by comparing the generated electronic medical record with the revised electronic medical record.

[0017] Furthermore, the acoustic model learning module obtains voice information, distinguishes the person corresponding to the voice information based on voice features, and establishes a preliminary acoustic model; at the same time, obtains the identity information of the person in the voice information; then performs semantic reasoning based on the text information converted by the voice segmentation system, and matches the voice information with the identity information of the person; combines the historical medical voice information stored in the medical voice database for learning, updates the personalized voice features of the person in the voice information, and generates a personalized acoustic model.

[0018] Furthermore, the language model learning module establishes a language model based on historical diagnosis and treatment voice information and historical doctor-patient dialogue texts, combined with a medical terminology knowledge base and a dialect voice library, according to the time information and identity information corresponding to the voice information, and updates the personalized features in the language model by comparing the oral medical record text with the revised medical record text.

[0019] Furthermore, the writing model learning module establishes a personalized writing model based on the medical records of the electronic medical record system, the standard medical record template library and the medical terminology knowledge base; and updates and optimizes the writing model based on the modification records of the medical records recorded in the input sampling system.

[0020] Compared with the prior art, the present invention has the following beneficial effects: the electronic medical record voice interaction device based on machine learning provided by the present invention, the voice segmentation system combines the three data of doctor-patient conversation content, doctor-patient identity information and doctor-patient personalized voice features to more accurately identify the identity of the interlocutor, facilitate distinguishing the third party, eliminate interference and realize automatic voice entry into the medical record; the machine learning system establishes the doctor's personalized acoustic model and language model, more accurately identifies the doctor's speech content, and avoids entry errors caused by the doctor's different accents, word usage, and abbreviation habits; the machine learning system automatically learns without the doctor's additional evaluation results. As the use time increases, the acoustic model for sound recognition, the voice model for language organization and the writing model for medical record generation are more accurate; the doctor's consultation efficiency is improved and the accuracy of electronic medical records is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of the structure of an electronic medical record voice interaction device based on machine learning according to an embodiment of the present invention;

[0022] Figure 2 A flowchart of a speech recognition module according to an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a speech recognition module according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart of the NLP module of an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the NLP module of an embodiment of the present invention;

[0026] Figure 6 This is a flowchart of the acoustic model learning module of an embodiment of the present invention;

[0027] Figure 7 A diagram showing the learning process of an acoustic model learning module according to an embodiment of the present invention;

[0028] Figure 8 This is a working diagram of a language model learning module according to an embodiment of the present invention;

[0029] Fig. 9 A learning process diagram of a language model learning module according to an embodiment of the present invention;

[0030] Fig.10This is a workflow diagram of a writing model learning module according to an embodiment of the present invention.

[0031] In the figure:

[0032] 1. Client; 2. Server; 3. Voice input device; 4. Electronic medical record system; 5. Voice segmentation system; 6. Input sampling system; 7. Machine learning system. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0034] Figure 1 This is a schematic diagram of the structure of an electronic medical record voice interaction device based on machine learning according to an embodiment of the present invention.

[0035] See also Figure 1 The electronic medical record voice interaction device based on machine learning of the embodiment of the present invention includes a voice input device 3, a client 1 and a server 2, wherein the client 1 is deployed with an electronic medical record system 4, and the server 2 is deployed with a voice segmentation system 5, an input sampling system 6 and a machine learning system 7; the voice input device 3 collects voice information; the voice segmentation system 5 converts the voice information into text information and processes the text information to generate an electronic medical record and then sends it to the electronic medical record system 4; the electronic medical record system 4 displays the electronic medical record through the client 1 and corrects the electronic medical record; the input sampling system 6 collects the corrected electronic medical record and sends it to the voice segmentation system 5 for processing, and then provides it to the machine learning system 7 for model training; the machine learning system 7 provides the voice segmentation system 5 with an acoustic model, a language model and a writing model.

[0036] The server 2 is provided with a diagnosis and treatment voice database, a diagnosis and treatment text database, a dictionary, a medical terminology knowledge base, a dialect language base and a standard medical record template base.

[0037] The functions and working processes of each part are as follows:

[0038] 1. Voice Input Device

[0039] The voice input device 3 is connected to the client 1 device and / or the server 2 device (such as a computer device, a mobile device, etc.) via a connecting line (such as a USB protocol) to exchange data with the system running on the client 1 and / or the server 2. The voice input device 3 has a built-in microphone to collect conversations from both the doctor and the patient, and the collected data is processed by the voice segmentation system 5.

[0040] 2. Electronic Medical Record System

[0041] The electronic medical record system 4 is deployed on the client 1 and used by the doctor through the client 1 device. The doctor can check the medical records that have been entered for the patient, and can also check whether the electronic medical records automatically generated by the voice segmentation system 5 are accurate. The electronic medical record system 4 can simultaneously display the automatically generated electronic medical records and the corresponding doctor-patient dialogue text side by side, and support the doctor to make corrections based on the automatically generated electronic medical records.

[0042] The corrected electronic medical record will be transmitted to the input sampling system 6 of the server 2 through the interface, and will be processed by the speech segmentation system 5 to generate the learning materials required by the machine learning system 7.

[0043] During the consultation process, the electronic medical record system 4 monitors in real time whether there is any switching of the patient's medical record or change in identity information request, and provides a log analysis basis for the speech segmentation system 5 to analyze whether others have intruded into the conversation.

[0044] 3. Speech Segmentation System

[0045] The speech segmentation system 5 includes a speech recognition module and an NLP (Natural Language Processing) module;

[0046] The speech segmentation system 5 obtains the conversation speech information from the doctor and the patient from the speech input device 3, and converts the speech phonemes into the doctor-patient conversation text through the speech recognition module through vector conversion, acoustic model recognition, and language model recognition, wherein the acoustic model and the speech model are optimized according to the speech characteristics of the doctor and the patient based on the machine learning system 7. After the doctor-patient conversation text is processed by the NLP module, the identity information from the doctor and the patient is obtained through the interface, the keywords in the conversation are identified, and the electronic medical record is automatically filled in according to the keywords. Finally, the doctor can view the automatically entered electronic medical record in the electronic medical record system 4 after the consultation.

[0047] Speech Recognition Module

[0048] The function of the speech recognition module is to convert the acquired speech information into a doctor-patient dialogue text with identity information and time information. The specific process is as follows: Figure 2As shown: the speech recognition module obtains the speech of both the doctor and the patient from the speech input device 3, and cooperates with the identity information acquisition device such as the card swiping device connected to the client 1 interface to obtain the identity information of the doctor and the patient in the medical scene. After the speech phonemes are converted through vectors, they are input into the acoustic model and speech model that have been personalized and tuned by the machine learning system 7 to form a preliminary dialogue text. Combined with the speech features and the context of the conversation, the electronic medical record system 4 is used to monitor in real time whether there is a patient medical record switch or a change in identity information request, and analyze whether there is a change in the dialogue characters, such as whether the previous patient has finished the consultation, whether there are any patients or medical staff who suddenly break in, etc., and remove the dialogue content that is not related to the consultation inserted in the text. Finally, a doctor-patient dialogue text with time information and identity information is generated.

[0049] The specific working principle of the language recognition module is as follows Figure 3 As shown: the diagnosis and treatment voice database stores historical diagnosis and treatment voice information, and the acoustic model is obtained by the machine learning system 7 after obtaining the historical diagnosis and treatment voice information from the diagnosis and treatment voice database, extracting the voice features and training the model; the diagnosis and treatment text database stores historical doctor-patient conversation texts and oral medical record texts, and the language model is obtained by the machine learning system 7 after obtaining the historical doctor-patient conversation texts and oral medical record texts from the diagnosis and treatment text database and training the model; the dictionary connects the acoustic model and the voice model, and stores the correspondence between voice and text.

[0050] NLP Module

[0051] The function of the NLP module is to process the dialogue text output by the speech recognition module and automatically fill it into the electronic medical record. The specific process is as follows: Figure 4 As shown: the NLP module obtains the doctor-patient dialogue text output by the speech recognition module, and then combines the medical terminology database in the machine learning system 7 to segment the doctor-patient dialogue text, identify keywords, obtain the original key dialogue material, and convert it into the text of the doctor's daily writing style according to the writing model personalized by the machine learning system 7. At the same time, some keywords are converted into standard entries in the dictionary according to the requirements of electronic medical record writing. The generated text and dictionary are automatically filled into various fields of the electronic medical record, including the chief complaint, current medical history, past medical history, and other objective fact description parts of the medical record that do not require the doctor to make a judgment to draw a conclusion. Finally, when the doctor is finished with the consultation, he confirms the automatically filled electronic medical record by checking the electronic medical record system 4 and makes corrections and adjustments.

[0052] In addition, the input sampling system 6 provides the revised electronic medical records to the machine learning system 7 as learning materials, which also need to be provided after word segmentation by the NLP module.

[0053] The specific working principle of the NLP module is as follows Figure 5As shown in the figure: First, the doctor-patient conversation text is preprocessed and segmented through the diagnosis and treatment text database and medical terminology vocabulary; then the data is subjected to medical named entity recognition through the rule set set by the rule description language; then, according to the entropy-expanded term extraction algorithm, it is determined whether the recognized keyword is a single keyword or a compound keyword; finally, it is filtered through the general vocabulary to remove the conventional keywords, and then the label is extracted.

[0054] In the label extraction process, we first extract labels and construct label vectors through the medical terminology database and combine them with the keywords obtained in entity recognition, and then perform quantification operations. Then, we calculate the correlation between labels and descriptions, and between labels, and construct patient cases. Finally, we use the writing model to automatically fill in the electronic medical records.

[0055] 4. Input sampling system

[0056] The input sampling system 6 samples the content entered by the doctor, including the patient medical record content entered by the doctor and the content revised by the doctor after automatic generation (including the text copied and pasted when comparing the original text in the electronic medical record system 4), and adds the corresponding creation time and modification time as important marks for the machine learning system 7 to conduct correlation analysis and learning of text and speech. The sampled text needs to be segmented by the NLP module of the speech segmentation system 5 and then provided to the machine learning system 7.

[0057] 5. Machine Learning System

[0058] The machine learning system 7 provides personalized models for doctors and patients for the speech segmentation system 5. The training of the models is based on voice information, the text of the doctor-patient dialogue after sound conversion, and the electronic medical record data entered in the electronic medical record. The personalized model includes an acoustic model, a language model, and a writing model. All models adopt the principle of silent automatic learning to ensure that the cost of manual intervention during learning is minimized. The machine learning system 7 includes an acoustic model learning module, a language model learning module, and a writing model learning module.

[0059] Acoustic model learning module

[0060] The acoustic model learning module uses the voice information collected by the voice input device 3 to generate personalized acoustic models for doctors and patients who have been treated, and provides personalized acoustic models when the voice recognition module performs acoustic recognition. The workflow is as follows: Figure 6As shown: the acoustic model learning module obtains the voice information of both the doctor and the patient collected by the voice input device 3, first distinguishes which different people are in the voice information, and establishes a preliminary acoustic model. At the same time, the character information in the medical scene is obtained, for example, the identity information of the corresponding doctor is obtained based on the account logged in by the attending doctor in the current medical scene, and the identity information of the corresponding patient is obtained based on the patient's medical card swiping record on the card swiping device and the electronic medical record switching record. Then, the doctor-patient dialogue text output by the voice recognition module is semantically inferred to automatically determine the specific identity of the person, and at the same time establish a voice history record. Then, the historical medical treatment voice information generated by the historical medical treatment is combined for learning, and the personalized features of the speaker's voice are added or updated to generate a personalized acoustic model. Adopting the principle of automated learning, as the amount of voice data collected for a specific person increases, the acoustic model for the specific person will become more and more accurate.

[0061] The specific learning process of the acoustic model learning module is as follows: Figure 7 As shown in the figure: first, the speech information is feature extracted and saved; then MFCC (Mel-scale Frequency Cepstral Coefficients, MFCC for short) is used to extract speech feature parameters; then the speech features extracted by MFCC method are trained with single factors and multiple factors; then the deep neural network training is performed after forced alignment using the Viterbi algorithm; finally, the processed features are trained to obtain an acoustic model based on the DNN neural network.

[0062] Language model learning module

[0063] The language model learning module compares the generated doctor-patient dialogue text with the final results of the doctor's modification of the automatically generated electronic medical record, comparison of the original text, and copy and paste operations, and then automatically adjusts the language model. Its workflow is as follows: Figure 8 As shown: The language model learning module first uses the generated historical doctor-patient dialogue text and historical diagnosis and treatment voice information, combined with the medical terminology knowledge base and dialect voice library built into the machine learning system 7, to distinguish the voices of different characters in the voice according to the timeline and identity information, and establish a preliminary language model. Then, the generated doctor-patient dialogue text is compared with the final results of the doctor's modification operations on the automatically generated electronic medical records, viewing the original text for comparison, and copying and pasting operations to update the personalized features in the language model. Using automated supervised learning, as the doctor's corrections to the medical records and comparisons with the original text increase, the language model will become more and more accurate. The specific learning process of the language model learning module is as follows: Fig. 9 shown.

[0064] Writing Model Learning Module

[0065] The writing model learning module uses doctors’ modification records of automatically generated electronic medical records to automatically supervise learning and automatically adjust the writing model. Fig.10 As shown: the writing model learning module first uses the electronic medical records written by the doctor, the standard medical record template library and the medical terminology knowledge base in the machine learning system 7 to establish a personalized preliminary writing model for the doctor. The personalized writing module is updated using the doctor's modification records of the automatically generated electronic medical records captured by the input sampling system 6. With the use of automated supervised learning, the writing model will become more and more accurate as the doctor's modification behavior on the medical records increases.

[0066] In the actual use of the electronic medical record voice interaction device based on machine learning in the embodiment of the present invention, before the consultation, the doctor inserts the voice input device 3 and logs into the account in the electronic medical record system 4 to start the consultation. The voice input device 3 starts to collect voice information. When the patient is seeing the doctor, the patient's identity information is confirmed by the card swiping device. The voice segmentation system 5 analyzes the voice information of both the doctor and the patient, converts it into a doctor-patient dialogue text, and generates an electronic medical record that meets the requirements based on the doctor's personalized writing style. After the consultation, the doctor modifies the electronic medical record. In this process, the input sampling system 6 records and samples the content and modification records entered by the doctor, and provides them to the machine learning system 7 for further automatic tuning of the relevant models of voice recognition and medical record generation.

[0067] During the implementation process, a period of tuning can be carried out, and only the voice input device 3 is inserted to capture the voice information of both the doctor and the patient. The doctor still manually writes the medical record on the electronic medical record system 4, and the input results are sampled by the input sampling system 6, and the machine learning system 7 automatically trains the relevant model in advance. After learning more than ten cases, the function of automatically generating electronic medical records can be opened.

[0068] Preferably, an identity authentication system may be added, and in addition to voice input, a key interaction module, a camera module, etc. may be added to the external voice input hardware device.

[0069] Physician's identity authentication no longer needs to use the electronic medical record's identity authentication interface to log in. Voice and voiceprint authentication can be used to log in. At the same time, a voice recognition library is also established for all patients who visit the hospital. Combined with cameras to monitor the entry and exit of people in the room, the situation of misrecognition of medical conversations caused by other people accidentally intruding during the consultation can be further reduced, and the accuracy of medical records can be greatly improved.

[0070] The physician may use a key interaction module or natural language instructions (for example, let me see your previous test report) to command the electronic medical record system 4 to quickly open a certain type or a certain medical record of a returning patient, thereby improving the user experience.

[0071] To sum up, in the electronic medical record voice interaction device based on machine learning in the embodiment of the present invention, the voice segmentation system 5 combines the three aspects of data, namely, the doctor-patient conversation content, the doctor-patient identity information, and the doctor-patient personalized voice features, to more accurately identify the identity of the interlocutor, facilitate the distinction of third parties, and eliminate interference to realize automatic voice entry into medical records; the machine learning system 7 establishes a personalized acoustic model and language model for doctors, to more accurately identify the doctor's speech content, and avoid entry errors caused by the doctors' different accents, words, and abbreviation habits; the machine learning system 7 learns automatically without the doctor's additional evaluation of the results. As the use time increases, the acoustic model for sound recognition, the voice model for language organization, and the writing model for medical record generation are more accurate; the doctor's consultation efficiency is improved and the accuracy of electronic medical records is improved.

[0072] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. An electronic medical record voice interaction device based on machine learning, characterized in that: It includes a voice input device, a client and a server, wherein the client is equipped with an electronic medical record system, and the server is equipped with a voice segmentation system, an input sampling system and a machine learning system; The voice input device collects voice information; The speech segmentation system converts the speech information into text information and processes the text information to generate an electronic medical record and then sends it to the electronic medical record system; The electronic medical record system displays the electronic medical record through the client and makes corrections to the electronic medical record; The input sampling system collects and modifies the electronic medical records, sends them to the speech segmentation system for processing, and then provides them to the machine learning system for model training; The machine learning system provides an acoustic model, a language model and a writing model for the speech segmentation system; The server is provided with a diagnosis and treatment voice database, a diagnosis and treatment text database, a dictionary, a medical terminology knowledge base, a dialect language base and a standard medical record template base; The speech segmentation system includes a speech recognition module and an NLP module; the speech recognition module obtains speech information through a speech input device, and the speech information includes a medical consultation dialogue and an oral medical record; the speech recognition module obtains the identity information of both the doctor and the patient through an identity information acquisition device connected to the client; the speech recognition module converts the speech information of the medical consultation dialogue into a doctor-patient dialogue text with identity information and time information according to the identity information, and the speech recognition module converts the oral medical record speech information into an oral medical record text with identity information and time information; the NLP module processes the doctor-patient dialogue text and / or the oral medical record text to generate an electronic medical record and then sends it to the electronic medical record system; The speech recognition module converts the speech information into sound vectors and then sequentially inputs them into the acoustic model for acoustic recognition and the language model for language recognition to generate a preliminary dialogue text; at the same time, the client monitors whether there is a change in identity information request through the connected identity information acquisition device, and the electronic medical record system monitors whether there is a patient medical record switch, and determines whether there is a change in the dialogue character in combination with the preliminary dialogue text, identifies and removes the dialogue content that is inserted in the middle and is not related to the consultation in the preliminary dialogue text, and generates a doctor-patient dialogue text with identity information and time information; The diagnosis and treatment voice database stores historical diagnosis and treatment voice information, and the acoustic model is obtained by the machine learning system after obtaining the historical diagnosis and treatment voice information from the diagnosis and treatment voice database and extracting voice features for model training; the diagnosis and treatment text database stores historical doctor-patient conversation texts and oral medical record texts, and the language model is obtained by the machine learning system after obtaining the historical doctor-patient conversation texts and oral medical record texts from the diagnosis and treatment text database and performing model training; the dictionary connects the acoustic model and the voice model, and stores the correspondence between voice and text; The NLP module performs word segmentation on the doctor-patient dialogue text to identify keywords in the doctor-patient dialogue text, converts the keywords into text in the daily writing style through a writing model, fills in the objective fact description part of the electronic medical record in combination with the electronic medical record writing requirements, and sends it to the electronic medical record system; the objective fact description part of the electronic medical record includes the chief complaint part, the current medical history part, and the past medical history part; The NLP module preprocesses and segments the doctor-patient dialogue text and obtains keywords based on the diagnosis and treatment text database and the medical terminology knowledge base; Then, the medical named entities are recognized through the rule set set by the rule description language; then, the term extraction algorithm with entropy expansion is used to determine whether the recognized keyword is a single keyword or a compound keyword; finally, the electronic medical record labels are extracted and the label vector is constructed after filtering through the common vocabulary and removing the conventional keywords, and the various parts of the electronic medical record are filled in according to the electronic medical record labels through the writing model.

2. The electronic medical record voice interaction device based on machine learning as claimed in claim 1, characterized in that: The machine learning system includes an acoustic model learning module, a language model learning module and a writing model learning module; the medical terminology knowledge base, the dialect language base and the standard medical record template base are set in the machine learning system; The acoustic model learning module generates a personalized acoustic model associated with identity information using the voice information collected by the voice input device; The language model learning module adjusts the language model by comparing the generated doctor-patient dialogue text with the revised electronic medical record; The writing model learning module adjusts the writing model by comparing the generated electronic medical record with the revised electronic medical record.

3. The electronic medical record voice interaction device based on machine learning as claimed in claim 2, characterized in that: The acoustic model learning module obtains voice information, distinguishes the person corresponding to the voice information according to the voice features, and establishes a preliminary acoustic model; at the same time, it obtains the identity information of the person in the voice information; then performs semantic reasoning based on the text information converted by the voice segmentation system, and matches the voice information with the identity information of the person; combines the historical medical voice information stored in the medical voice database for learning, updates the personalized voice features of the person in the voice information, and generates a personalized acoustic model.

4. The electronic medical record voice interaction device based on machine learning as claimed in claim 2, characterized in that: The language model learning module establishes a language model based on historical diagnosis and treatment voice information and historical doctor-patient dialogue texts, combined with a medical terminology knowledge base and a dialect voice library, according to the time information and identity information corresponding to the voice information, and updates the personalized features in the language model by comparing the oral medical record text with the revised medical record text.

5. The electronic medical record voice interaction device based on machine learning as claimed in claim 2, characterized in that: The writing model learning module establishes a personalized writing model based on the medical records of the electronic medical record system, the standard medical record template library and the medical terminology knowledge base; and updates and optimizes the writing model based on the modification records of the medical records recorded in the input sampling system.

Images