A method and device for real-time guidance of doctor-patient dialogue based on multimodal quality perception

Abstract

This invention discloses a method and apparatus for real-time guidance of doctor-patient dialogue based on multimodal quality perception. The method includes: acquiring a real-time voice stream containing diagnosis-related voice interaction information between the doctor and patient; recognizing the real-time voice stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the weight of the field matching the first key field; calculating a completeness score based on the field weights; the completeness score representing the proportion of the first key field in the medical record text; acquiring a second key field when the completeness score is less than a preset threshold; the second key field being any missing field in the medical record text other than the first key field; and generating the medical record text based on the first and second key fields. This technical solution possesses intelligent guidance capabilities and can achieve medical record quality assessment and standardization.

Description

Technical Field

[0001] This invention relates to the fields of medical informatics and artificial intelligence, and in particular to a method and apparatus for real-time guidance of doctor-patient dialogue based on multimodal quality perception. Background Technology

[0002] In the current era of rapid development in medical informatization, electronic medical records, as the core carrier of medical data, directly impact the conduct of clinical diagnosis and treatment and the standardization of medical services through their writing efficiency and quality. To address the problems of low efficiency and the significant time doctors spend on manual medical record entry, voice-based medical record systems are gradually being applied in clinical settings, becoming an important tool for freeing doctors' hands and improving the efficiency of medical record writing, especially in high-frequency scenarios such as outpatient and inpatient care.

[0003] Currently, voice medical record systems in the medical field are mainly built upon three core technology solutions, forming a basic process from voice acquisition to medical record generation: first, speech recognition and separation technology; second, natural language processing (NLP) technology; and third, large language models (LLMs) applications.

[0004] However, existing voice-based medical record systems still have many prominent problems and shortcomings in practical clinical applications, especially in specialized departments such as psychiatry and emergency medicine where high accuracy and timeliness of consultation are required. These shortcomings make it difficult to meet the actual needs of clinical diagnosis and treatment. Specifically, the systems exhibit a passive recording mode and lack proactive guidance capabilities. Furthermore, medical record quality assessment is often delayed. Current systems typically assess the completeness and standardization of medical records only after the entire consultation process has concluded. Summary of the Invention

[0005] This invention provides a method and device for real-time guidance of doctor-patient dialogue based on multimodal quality perception, which has intelligent guidance capabilities and can realize medical record quality assessment and standardization.

[0006] According to one aspect of the present invention, a method for real-time guidance of doctor-patient dialogue based on multimodal quality perception is provided, the method comprising: Acquire real-time audio stream; wherein the real-time audio stream includes diagnosis-related audio interaction information between the doctor and the patient; The real-time audio stream is identified to obtain the first key field in the medical record text; The first key field is matched using a preset medical record knowledge graph to determine the field weight that matches the first key field. Based on the field weights, a completeness score is calculated; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text; When the completeness score is less than a preset threshold, the second key field is obtained; wherein, the second key field is the missing field in the medical record text other than the first key field; Based on the first key field and the second key field, generate the medical record text.

[0007] According to another aspect of the present invention, a real-time guidance device for doctor-patient dialogue based on multimodal quality perception is provided, the device comprising: A real-time voice stream acquisition module is used to acquire real-time voice streams; wherein, the real-time voice streams include diagnosis-related voice interaction information between doctors and patients; The first key field acquisition module is used to identify the real-time voice stream and obtain the first key field in the medical record text; The field weight determination module is used to match the first key field with a preset medical record knowledge graph and determine the field weight that matches the first key field. The completeness score calculation module is used to calculate the completeness score based on the field weights; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text; The second key field acquisition module is used to acquire a second key field when the completeness score is less than a preset threshold; wherein, the second key field is a missing field in the medical record text other than the first key field; The medical record text generation module is used to generate medical record text based on the first key field and the second key field.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute a real-time guidance method for doctor-patient dialogue based on multimodal quality perception as described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement a real-time guidance method for doctor-patient dialogue based on multimodal quality perception as described in any embodiment of the present invention.

[0010] The technical solution of this invention involves: acquiring a real-time audio stream; recognizing the real-time audio stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the weight of the field matching the first key field; calculating a completeness score based on the field weights; acquiring a second key field when the completeness score is less than a preset threshold; and generating the medical record text based on the first and second key fields. This technical solution possesses intelligent guidance capabilities and can achieve medical record quality assessment and standardization.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a real-time guidance method for doctor-patient dialogue based on multimodal quality perception according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the doctor-patient dialogue system provided in Embodiment 1 of this application; Figure 3 A flowchart of another real-time guidance method for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 2 of the present invention; Figure 4 A flowchart of another real-time guidance method for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 3 of the present invention; Figure 5 This is a schematic diagram of a real-time guidance process for doctor-patient dialogue based on multimodal quality perception, provided in Embodiment 4 of the present invention. Figure 6 This is a schematic diagram of another real-time guidance process for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 5 of the present invention; Figure 7 This is a schematic diagram of another real-time guidance and recognition process for doctor-patient dialogue based on multimodal quality perception provided in Embodiment Six of the present invention; Figure 8 This is a schematic diagram of the structure of a real-time doctor-patient dialogue guidance device based on multimodal quality perception provided in Embodiment 7 of the present invention; Figure 9This is a schematic diagram of the structure of an electronic device that implements a real-time guidance method for doctor-patient dialogue based on multimodal quality perception, according to an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Example 1 Figure 1 This is a flowchart of a real-time doctor-patient dialogue guidance method based on multimodal quality perception according to Embodiment 1 of the present invention. This embodiment is applicable to real-time doctor-patient dialogue guidance and medical record text completion. The method can be executed by a real-time doctor-patient dialogue guidance device based on multimodal quality perception. This device can be implemented in hardware and / or software and can be configured in a device. For example, the device can be a backend server or other device with communication and computing capabilities. Figure 1 As shown, the method includes: S110. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0017] In this solution, real-time voice streams are acquired using voice acquisition devices. These streams contain diagnosis-related voice interactions between doctors and patients. For example, the real-time voice stream could include audio information such as doctors asking patients about their conditions, patients describing symptoms, and doctors providing diagnostic opinions or treatment suggestions.

[0018] In this embodiment, Figure 2 This is a schematic diagram of the doctor-patient dialogue system provided in Embodiment 1 of this application, as shown below. Figure 2 As shown, the system is driven by real-time voice streams. After recognition and separation, the signals are generated by the core real-time quality assessment engine, intelligent guidance and prompting module, and finally output through LLM structured output.

[0019] S120. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0020] Among them, medical record text is the diagnosis-related text data generated after processing the voice interaction information between doctors and patients.

[0021] In this scheme, the first key field is a key data item extracted from the real-time voice stream to represent the core information of diagnosis and treatment. For example, the first key field includes: the patient's symptoms are headache and dizziness; the past medical history is no history of hypertension; and the diagnosis is tension headache.

[0022] In this embodiment, speech recognition and separation technology, natural language processing technology, large language model, binaural acquisition and voiceprint recognition technology are used to identify the real-time speech stream and obtain the first key field in the medical record text.

[0023] Specifically, the system integrates dual-channel acquisition and voiceprint recognition technology at the bottom layer. By analyzing the voiceprint features of pre-sampled doctors, the real-time voice stream can be split into two text channels, namely the doctor's trajectory and the patient's trajectory, at the beginning of the consultation, thereby ensuring that the subsequent real-time quality assessment engine can accurately identify the speaker's identity.

[0024] S130. Use a preset medical record knowledge graph to match the first key field and determine the field weight that matches the first key field.

[0025] In this scheme, the medical record knowledge graph is constructed based on the ICD-10 standard and clinical pathways of specific specialties (such as psychiatry). Entities include symptoms (such as "hallucinations"), triggers, duration, severity, etc. Relationships include "accompaniment relationships" (such as hallucinations often accompanied by delusions) and "exclusion relationships" (such as the real-time opposition between manic and depressive symptoms). Attributes define "required weights" (mandatory, important, general) for each symptom.

[0026] The field weights correspond to the attributes in the medical record knowledge graph.

[0027] Furthermore, the first key field is matched with the medical record knowledge graph to obtain the field weights that match the first key field.

[0028] S140. Based on the field weights, calculate the completeness score; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0029] In this scheme, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0030] Specifically, the completeness score is obtained by calculating the proportion of each field's weight in the total weight of all key fields.

[0031] In this embodiment, the doctor-patient dialogue content is monitored in real time, and the completeness score is automatically updated when field keywords or semantic overlap is detected.

[0032] S150. When the completeness score is less than a preset threshold, obtain the second key field; wherein, the second key field is the missing field in the medical record text other than the first key field.

[0033] The threshold is set according to the needs of doctor-patient dialogue processing.

[0034] In this embodiment, the second key field is the missing field in the medical record text other than the first key field.

[0035] Specifically, when the completeness score is less than the preset threshold, it indicates that the medical record does not meet the submission criteria and the second key field needs to be obtained through interaction.

[0036] In this scheme, when the completeness score is greater than or equal to the preset threshold, it indicates that the medical record has met the submission criteria and the medical record text can be directly generated based on the first key field.

[0037] S160. Generate medical record text based on the first key field and the second key field.

[0038] Specifically, the first and second key fields are matched to the fill positions of each field in the preset template to generate medical record text.

[0039] The technical solution of this invention involves: acquiring a real-time audio stream; recognizing the real-time audio stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the weight of the field matching the first key field; calculating a completeness score based on the field weights; acquiring a second key field when the completeness score is less than a preset threshold; and generating the medical record text based on the first and second key fields. By implementing this technical solution, intelligent guidance capabilities are achieved, enabling medical record quality assessment and standardization.

[0040] Example 2 Figure 3This is a flowchart of another real-time guidance method for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the field weight determination process. Figure 3 As shown, the method includes: S310. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0041] S320. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0042] S330. Match the first key field with entities in the preset medical record knowledge graph to determine the entity corresponding to the first key field in the medical record knowledge graph.

[0043] In this plan, such as Figure 2 As shown, the system uses the medical record knowledge graph in the knowledge graph matcher to match the first key field. Specifically, the system attaches the first key field to a node in the medical record knowledge graph to determine the entity corresponding to the first key field in the medical record knowledge graph.

[0044] S340. Extract the attribute values ​​corresponding to the entity.

[0045] Furthermore, the system extracts the attribute values ​​corresponding to entities based on the correspondence between entities and attributes in the medical record knowledge graph.

[0046] S350. Determine the field weight that matches the first key field based on the attribute value.

[0047] In this approach, the attribute value is determined as the field weight that matches the first key field. For example, in a psychiatric consultation, a field weight that matches a history of allergies is considered important; a field weight that matches personal hobbies is considered average.

[0048] S360. Based on the field weights, a completeness score is calculated; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0049] S370. When the completeness score is less than a preset threshold, obtain the second key field; wherein, the second key field is the missing field in the medical record text other than the first key field.

[0050] S380. Generate medical record text based on the first key field and the second key field.

[0051] The technical solution of this invention involves acquiring a real-time voice stream; recognizing the real-time voice stream to obtain a first key field in the medical record text; matching the first key field with entities in a medical record knowledge graph to determine the field weight matching the first key field; calculating a completeness score based on the field weight; acquiring a second key field when the completeness score is less than a preset threshold; and generating the medical record text based on the first and second key fields. By implementing this technical solution and constructing a multi-medical record knowledge graph, real-time quantitative evaluation of medical record completeness is achieved, providing a reliable basis for judgment for intelligent guidance and dialogue interaction control in the system, and improving the standardization and completeness of medical record generation.

[0052] Example 3 Figure 4 This is a flowchart of another real-time guidance method for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is described in detail as a process for calculating the completeness score. Figure 4 As shown, the method includes: S410. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0053] S420. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0054] S430. Use a preset medical record knowledge graph to match the first key field and determine the field weight that matches the first key field.

[0055] S440. Sum the products of the field weights and the content quality coefficients corresponding to the first key field to obtain the first score.

[0056] In this scheme, the content quality coefficient ranges from 0 to 1 and is calculated by the semantic coverage algorithm. For example, if a doctor only mentions the name of a symptom (such as hallucinations), the content quality coefficient is 0.4; if the nature, frequency, and duration are further mentioned, the content quality coefficient is 1.

[0057] Specifically, the first score is determined using the following formula; .

[0058] S450. Sum the field weights corresponding to all entities in the medical record knowledge graph to obtain the second score.

[0059] In this scheme, the second score is calculated using the following formula; .

[0060] S460. Divide the first score by the second score to obtain the completeness score.

[0061] Furthermore, the completeness score is calculated using the following formula; .

[0062] S470. When the completeness score is less than a preset threshold, obtain the second key field; wherein, the second key field is the missing field in the medical record text other than the first key field.

[0063] S480. Generate medical record text based on the first key field and the second key field.

[0064] The technical solution of this invention involves: acquiring a real-time speech stream; recognizing the real-time speech stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the field weights that match the first key field; summing the products of the field weights and the content quality coefficients corresponding to the first key field to obtain a first score; summing the field weights corresponding to all entities in the medical record knowledge graph to obtain a second score; dividing the first score by the second score to obtain a completeness score; when the completeness score is less than a preset threshold, acquiring a second key field; and generating the medical record text based on the first and second key fields. By executing this technical solution, real-time assessment of medical record completeness is achieved through speech recognition, knowledge graph matching, and weight quantification calculation, intelligently guiding the supplementation of key fields, and effectively improving the accuracy, completeness, and intelligence level of medical record generation.

[0065] Example 4 Figure 5 This is a schematic diagram of a real-time doctor-patient dialogue guidance process based on multimodal quality perception, provided in Embodiment 4 of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the missing field identification process. Figure 5 As shown, the method includes: S510. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0066] S520. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0067] S530. Use a preset medical record knowledge graph to match the first key field and determine the field weight that matches the first key field.

[0068] S540. Based on the field weights, calculate the completeness score; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0069] S550. Determine the third key field; wherein the third key field is the difference between the first key field and all entities in the medical record knowledge graph.

[0070] In this plan, such as Figure 2 As shown, the missing item identifier in the real-time quality assessment engine is used to determine the third key field.

[0071] Specifically, by calculating the difference between the first key field and all key fields in the medical record knowledge graph, and combining this with the content completeness score, the missing fields that urgently need to be filled are identified.

[0072] S560. When the completeness score is less than a preset threshold, generate inquiry suggestions based on the third key field and the pre-determined question template.

[0073] Specifically, such as Figure 2 As shown, based on the spark prompting mechanism in the intelligent guidance and prompting module, query suggestions are generated for the third key field according to the preset question template and real-time voice stream. For example, if a hallucination description is detected, the query suggestion could be whether the voice is in one's ear or in one's head.

[0074] S570. The inquiry suggestion is pushed to prompt the patient to provide additional information.

[0075] Furthermore, after generating inquiry suggestions, these suggestions are pushed to patients to prompt them to supplement their information based on the suggestions.

[0076] S580: Receive information from the patient based on the inquiry suggestions to obtain the second key field.

[0077] In this approach, the second key field is obtained by identifying the information provided by the patient based on the inquiry and suggestion feedback.

[0078] S590. Generate medical record text based on the first key field and the second key field.

[0079] The technical solution of this invention involves acquiring a real-time speech stream; recognizing the real-time speech stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the weight of the field matching the first key field; calculating a completeness score based on the field weight; generating inquiry suggestions based on a third key field and a pre-determined question template when the completeness score is less than a preset threshold; pushing the inquiry suggestions to prompt the patient to supplement information; receiving feedback from the patient based on the inquiry suggestions to obtain a second key field; and generating the medical record text based on the first and second key fields. By executing this technical solution, through real-time speech recognition, knowledge graph matching, and completeness quantification assessment, targeted inquiry suggestions are intelligently generated and information supplementation is guided, significantly improving the real-time performance, completeness, and interactive intelligence of medical record generation. The clinical consultation logic is transformed into an executable algorithm framework, enabling real-time identification of information gaps and dynamic follow-up questions during dialogue, transforming the system from passive recording to active assistance, effectively improving the standardization and completeness of consultations and reducing the burden of subsequent data entry.

[0080] Example 5 Figure 6 This is a schematic diagram of another real-time guidance process for doctor-patient dialogue based on multimodal quality perception provided in Embodiment 5 of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the medical record text generation process. Figure 6 As shown, the method includes: S610. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0081] S620. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0082] S630. Use a preset medical record knowledge graph to match the first key field and determine the field weight that matches the first key field.

[0083] S640. Based on the field weights, calculate the completeness score; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0084] S650. When the completeness score is less than a preset threshold, obtain the second key field; wherein, the second key field is the missing field in the medical record text other than the first key field.

[0085] S660. Determine the medical record template that matches the first key field.

[0086] In this plan, such as Figure 2As shown, by using a template dynamic adapter, the medical record template that matches the first key field is determined from the template database, so as to realize the on-demand generation of consultation lists.

[0087] S670. Identify the fill position of each field in the medical record template.

[0088] Specifically, the medical record template is identified to determine the filling position of each field in the medical record template.

[0089] S680. Match the first key field and the second key field to the filling position to generate medical record text.

[0090] In this solution, based on the capabilities of a large language model, the first and second key fields are matched to the fill positions to generate medical record text.

[0091] The technical solution of this invention involves: acquiring a real-time audio stream; recognizing the real-time audio stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the field weights that match the first key field; calculating a completeness score based on the field weights; acquiring a second key field when the completeness score is less than a preset threshold; determining a medical record template that matches the first key field; identifying the filling positions of each field in the medical record template; and matching the first and second key fields to the filling positions to generate the medical record text. By implementing this technical solution, intelligent guidance capabilities are achieved, enabling medical record quality assessment and standardization.

[0092] Example 6 Figure 7 This is a schematic diagram of another real-time guidance process for doctor-patient dialogue based on multimodal quality perception provided in Embodiment Six of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the medical record text generation process. Figure 7 As shown, the method includes: S710. Obtain real-time voice stream; wherein, the real-time voice stream includes diagnosis-related voice interaction information between the doctor and the patient.

[0093] S720. Recognize the real-time voice stream to obtain the first key field in the medical record text.

[0094] S730. Use a preset medical record knowledge graph to match the first key field and determine the field weight that matches the first key field.

[0095] S740. Based on the field weights, calculate the completeness score; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text.

[0096] S750. When the completeness score is less than a preset threshold, obtain the second key field; wherein, the second key field is the missing field in the medical record text other than the first key field.

[0097] S760. Generate medical record text based on the first key field and the second key field.

[0098] S770. Filter out a fourth key field from the first key field and the second key field; wherein, the fourth key field is a field whose values ​​contradict each other; Specifically, such as Figure 2 As shown, the logic contradiction detector of the built-in rule engine compares the information conflicts between different fields in real time and filters out the fourth key field from the first key field and the second key field; for example, if the current medical history record shows hallucinations, but the mental examination field is marked as no hallucinations, then the relevant field is filtered out.

[0099] S780. Mark the fourth key field on the medical record text.

[0100] Furthermore, the fourth key field that is questionable or logically contradictory is highlighted to assist doctors in making the final confirmation.

[0101] The technical solution of this invention involves acquiring a real-time audio stream; recognizing the real-time audio stream to obtain a first key field in the medical record text; matching the first key field using a preset medical record knowledge graph to determine the field weight matching the first key field; calculating a completeness score based on the field weight; acquiring a second key field when the completeness score is less than a preset threshold; generating the medical record text based on the first and second key fields; selecting a fourth key field from the first and second key fields; and marking the fourth key field on the medical record text. By implementing this technical solution, medical record verification and supplementation are moved to the entire consultation process, significantly improving generation efficiency and medical record completeness, effectively reducing the omission rate of key information, ensuring medical safety, and possessing high interpretability and clinical credibility.

[0102] Example 7 Figure 8 This is a schematic diagram of a real-time doctor-patient dialogue guidance device based on multimodal quality perception, provided in Embodiment 7 of the present invention. Figure 8 As shown, the device includes: The real-time voice stream acquisition module 810 is used to acquire real-time voice streams; wherein, the real-time voice streams include diagnosis-related voice interaction information between doctors and patients; The first key field acquisition module 820 is used to identify the real-time voice stream and obtain the first key field in the medical record text. The field weight determination module 830 is used to match the first key field with a preset medical record knowledge graph and determine the field weight that matches the first key field. The completeness score calculation module 840 is used to calculate the completeness score based on the field weights; wherein the completeness score is used to characterize the proportion of the first key field in the medical record text; The second key field acquisition module 850 is used to acquire a second key field when the completeness score is less than a preset threshold; wherein, the second key field is a missing field in the medical record text other than the first key field; The medical record text generation module 860 is used to generate medical record text based on the first key field and the second key field.

[0103] Optional, the field weight determination module 830 is specifically used for: The first key field is matched with entities in the preset medical record knowledge graph to determine the entity corresponding to the first key field in the medical record knowledge graph. Extract the attribute values ​​corresponding to the entity; Based on the attribute value, determine the field weight that matches the first key field.

[0104] Optional, the completeness score calculation module 840 is specifically used for: The first score is obtained by summing the products of the field weights and the content quality coefficients corresponding to the first key field. The second score is obtained by summing the field weights corresponding to all entities in the medical record knowledge graph. Divide the first score by the second score to obtain the completeness score.

[0105] Optionally, the device further includes The third key field determination module is used to determine the third key field; wherein, the third key field is the difference between the first key field and all entities in the medical record knowledge graph.

[0106] Optional, the second key field retrieval module 850 is specifically used for: When the completeness score is less than a preset threshold, an inquiry suggestion is generated based on the third key field and a pre-determined question template; The suggested inquiry will be sent to the patient to prompt them to provide additional information; The system receives feedback from the patient based on the inquiry suggestions and obtains the second key field.

[0107] Optional, the medical record text generation module 860 is specifically used for: Identify the medical record template that matches the first key field; Identify the fill position of each field in the medical record template; Match the first key field and the second key field to the fill position to generate medical record text.

[0108] Optionally, the device further includes: The fourth key field filtering module is used to filter out a fourth key field from the first key field and the second key field; wherein, the fourth key field is a field whose values ​​contradict each other; A labeling template is used to label the fourth key field on the medical record text.

[0109] The real-time doctor-patient dialogue guidance device based on multimodal quality perception provided in this embodiment of the invention can execute the real-time doctor-patient dialogue guidance method based on multimodal quality perception provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0110] Example 8 Figure 9 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0111] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0112] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0113] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a real-time guidance method for doctor-patient dialogue based on multimodal quality awareness.

[0114] In some embodiments, a real-time guidance method for doctor-patient dialogue based on multimodal quality awareness can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the real-time guidance method for doctor-patient dialogue based on multimodal quality awareness described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a real-time guidance method for doctor-patient dialogue based on multimodal quality awareness by any other suitable means (e.g., by means of firmware).

[0115] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0118] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0119] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0120] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0121] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0122] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for real-time guidance of doctor-patient dialogue based on multimodal quality perception, characterized in that, include: Acquire real-time audio stream; wherein the real-time audio stream includes diagnosis-related audio interaction information between the doctor and the patient; The real-time audio stream is identified to obtain the first key field in the medical record text; The first key field is matched using a preset medical record knowledge graph to determine the field weight that matches the first key field. Based on the field weights, a completeness score is calculated; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text; When the completeness score is less than a preset threshold, the second key field is obtained; wherein, the second key field is the missing field in the medical record text other than the first key field; Based on the first key field and the second key field, generate the medical record text.

2. The method according to claim 1, characterized in that, The first key field is matched using a pre-defined medical record knowledge graph to determine the weight of the field that matches the first key field, including: The first key field is matched with entities in the preset medical record knowledge graph to determine the entity corresponding to the first key field in the medical record knowledge graph. Extract the attribute values ​​corresponding to the entity; Based on the attribute value, determine the field weight that matches the first key field.

3. The method according to claim 1, characterized in that, Based on the field weights, a completeness score is calculated, including: The first score is obtained by summing the products of the field weights and the content quality coefficients corresponding to the first key field. The second score is obtained by summing the field weights corresponding to all entities in the medical record knowledge graph. Divide the first score by the second score to obtain the completeness score.

4. The method according to claim 1, characterized in that, After calculating the completeness score based on the field weights, the method further includes... Determine the third key field; wherein the third key field is the difference between the first key field and all entities in the medical record knowledge graph.

5. The method according to claim 4, characterized in that, When the completeness score is less than a preset threshold, a second key field is obtained, including: When the completeness score is less than a preset threshold, an inquiry suggestion is generated based on the third key field and a pre-determined question template; The suggested inquiry will be sent to the patient to prompt them to provide additional information; The system receives feedback from the patient based on the inquiry suggestions and obtains the second key field.

6. The method according to claim 1, characterized in that, Based on the first key field and the second key field, generate medical record text, including: Identify the medical record template that matches the first key field; Identify the fill position of each field in the medical record template; Match the first key field and the second key field to the fill position to generate medical record text.

7. The method according to claim 1, characterized in that, After generating the medical record text based on the first key field and the second key field, the method further includes: A fourth key field is selected from the first and second key fields; wherein, the fourth key field is a field whose values ​​contradict each other. The fourth key field is marked on the medical record text.

8. A real-time guidance device for doctor-patient dialogue based on multimodal quality perception, characterized in that, include: A real-time voice stream acquisition module is used to acquire real-time voice streams; wherein, the real-time voice streams include diagnosis-related voice interaction information between doctors and patients; The first key field acquisition module is used to identify the real-time voice stream and obtain the first key field in the medical record text; The field weight determination module is used to match the first key field with a preset medical record knowledge graph and determine the field weight that matches the first key field. The completeness score calculation module is used to calculate the completeness score based on the field weights; wherein, the completeness score is used to characterize the proportion of the first key field in the medical record text; The second key field acquisition module is used to acquire a second key field when the completeness score is less than a preset threshold; wherein, the second key field is a missing field in the medical record text other than the first key field; The medical record text generation module is used to generate medical record text based on the first key field and the second key field.

9. The apparatus according to claim 8, characterized in that, The field weight determination module includes: The first key field is matched with entities in the preset medical record knowledge graph to determine the entity corresponding to the first key field in the medical record knowledge graph. Extract the attribute values ​​corresponding to the entity; Based on the attribute value, determine the field weight that matches the first key field.

10. The apparatus according to claim 8, characterized in that, The completeness score calculation module includes: The first score is obtained by summing the products of the field weights and the content quality coefficients corresponding to the first key field. The second score is obtained by summing the field weights corresponding to all entities in the medical record knowledge graph. Divide the first score by the second score to obtain the completeness score.

Images