Intelligent glasses-based diagnosis and treatment information interaction method and device

Abstract

The application discloses a diagnosis and treatment information interaction method and device based on smart glasses. The method comprises the following steps: when a doctor wears smart glasses to carry out bedside diagnosis and treatment, converting multi-modal bedside data collected by the smart glasses into a structured bedside context object; constructing a desensitized bedside context object into an evidence task package and sending the evidence task package to a task processing object; analyzing and checking a structured result returned by the task processing object to obtain an analysis result and a checking result; and displaying prompt information on the smart glasses according to the checking result. The application solves the technical problem of low efficiency of computer searching for medical information in the related art.

Description

Technical Field

[0001] This application relates to the field of medical informatics, and more specifically, to a method and device for interactive diagnostic and treatment information based on smart glasses. Background Technology

[0002] This section is intended to provide background or context for the content set forth in the claims or specification, and the content described herein is not acknowledged as prior art simply because it is included in this section.

[0003] In current bedside consultation scenarios (such as ward rounds and bedside consultations), doctors typically need to collect, record, and conduct preliminary analysis of diagnostic evidence, including the chief complaint, symptoms, signs, medical history, and medication use, without interrupting communication with the patient. Traditionally, this process heavily relies on manual recording by doctors and subsequent knowledge retrieval, specifically manifested as follows: During consultations, doctors convert bedside clinical information into text or structured fields through handwritten notes, temporary recordings, or manual entry into electronic medical records afterward. When encountering complex cases or requiring evidence-based medicine support (e.g., confirming guideline-recommended tests for a specific symptom, the latest risks associated with a drug, or diagnostic criteria for a rare disease), doctors often need to interrupt the consultation process. Relying on memory or returning to their office afterward, they can access medical literature databases, clinical guideline websites, or recently developed evidence-based medicine tools (such as clinical decision support tools based on large language models) via computer or mobile device, manually inputting the patient's key clinical characteristics to perform a search. The retrieved evidence (such as guideline clauses, literature abstracts, drug warnings, etc.) is then manually assessed by the doctor to determine its applicability to the current patient, and manually recorded or copied and pasted into the medical record. If the search results indicate insufficient information or contradictions, the doctor needs to schedule additional time to ask the patient or family further questions or consult other materials.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for interactive medical information based on smart glasses, so as to at least solve the technical problem of low efficiency in accessing medical information through computers in related technologies.

[0006] According to one aspect of the embodiments of this application, a method for interactive medical information based on smart glasses is provided, comprising: when a doctor wears smart glasses to perform bedside diagnosis and treatment, converting the multimodal bedside data collected by the smart glasses into a structured bedside context object; constructing the anonymized bedside context object into an evidence task package and sending it to a task processing object; parsing and verifying the structured result returned by the task processing object to obtain a parsing result and a verification result; and displaying a prompt message on the smart glasses according to the verification result.

[0007] According to another aspect of the embodiments of this application, a medical information interaction device based on smart glasses is also provided, comprising: a conversion unit, used to convert multimodal bedside data collected by the smart glasses into a structured bedside context object when a doctor wears smart glasses to perform bedside diagnosis and treatment; a sending unit, used to construct an evidence task package from the desensitized bedside context object and send it to a task processing object; a parsing and verification unit, used to parse and verify the structured result returned by the task processing object to obtain a parsing result and a verification result; and a display unit, used to display prompt information on the smart glasses according to the verification result.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the storage medium including a stored program that executes the above-described method when the program is run.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the above-described method through the computer program.

[0010] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the embodiments of the methods described above.

[0011] In this embodiment, when a doctor wears smart glasses to perform bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is converted into a structured bedside context object; the anonymized bedside context object is constructed into an evidence task package and sent to a task processing object; the structured result returned by the task processing object is parsed and verified to obtain the parsing result and the verification result; based on the verification result, a prompt message is displayed on the smart glasses, thereby solving the technical problem of low efficiency in accessing medical information via computer in related technologies. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an optional diagnostic information interaction method based on smart glasses according to an embodiment of this application; Figure 2 This is a schematic diagram of an optional smart glasses-based medical information interaction system according to an embodiment of this application; Figure 3 This is a schematic diagram of an optional evidence task construction and routing scheme according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional evidence analysis and verification scheme according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional reverse questioning and small screen prompt scheme according to an embodiment of this application; Figure 6 This is a schematic diagram of an optional smart glasses-based diagnostic information interaction device according to an embodiment of this application; and, Figure 7 This is a structural block diagram of a terminal according to an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows: General Naming Conventions: In this application, English and Chinese identifiers are primarily used to represent internal system data structures, field names, status names, or formula variables. Specifically, the suffixes are: _id (unique identifier), _type (type), _state (status), _score (score), _time (time), _ref or _refs (reference relationship), _version (version), _hash (hash digest), _flag (Boolean flag), _key (index key or deduplication key), _value (field value), _action (action object), _record (record object), and _job or _task (asynchronous task object).

[0016] The following are the definitions of the bedside closed-loop fields: bedside_stream represents the bedside data stream, bedside_context represents the bedside context object, evidence_task represents the evidence task object, field_filter_log represents the field filter log, return_schema represents the data structure returned by external evidence-based results, safety_check represents the safety check record, and audit_record represents the audit record.

[0017] External result field definitions: result_object represents the external result object, suggestion represents the suggestion, evidence represents the basis, condition represents the applicable conditions, gap represents the information gap, risk represents the risk warning, source_meta represents the source metadata, privacy_level represents the privacy level, local_hit_score represents the local hit score, and unstructured_result represents the external return result that does not meet the structural constraints.

[0018] Event and security status field explanations: audio_event, image_event, vital_event, and doctor_action represent audio events, image events, vital sign events, and doctor action events, respectively; P0 to P3 represent privacy processing levels, local_first indicates priority for local processing, external_allowed indicates that external calls are allowed, external_blocked indicates that external calls are prohibited, and reviewer_required indicates that manual review is required.

[0019] Figure 1 This is a flowchart of an optional diagnostic information interaction method based on smart glasses according to an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps: Step S110: When the doctor wears smart glasses to conduct bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is converted into a structured bedside context object. The multimodal bedside data includes multiple items such as audio and video data and patient medical data.

[0020] Optionally, a session identifier uniquely corresponding to the current medical session can be generated first, and a round identifier can be generated for each round of doctor-patient interaction; the target events in the multimodal bedside data can be converted into structured fields in the bedside context object, and each structured field can be associated with its source, round identifier, confidence level, and confirmation status. The target events include at least one of voice events, image events, vital sign events, and doctor action events; a list of locked fields can be set in the bedside context object, and the fields in the list of locked fields are prohibited from being automatically pruned or overwritten in the subsequent desensitization and task construction process.

[0021] Step S120: Construct the desensitized bedside context object into an evidence task package and send it to the task processing object (which may be a local database, a local evidence-based medicine agent, a remote database, or a remote evidence-based medicine agent).

[0022] Optionally, the fields in the bedside context object can be classified according to preset privacy classification rules. The privacy classification includes identity fields that are prohibited from being sent out, replaceable indirect identification fields, and clinically necessary fields. An evidence task package is generated using the clinically necessary fields and the transformed replaceable indirect identification fields (such as name replacement) in the bedside context object, and the excluded fields and their reasons for exclusion are recorded to generate a field filtering log. The evidence task package is sent to the task processing object according to the routing strategy. The routing strategy is selected from local routing priority, local routing only, external routing allowed, and external routing prohibited based on the task type, local knowledge base hit score, privacy level, and network status. The task processing object includes at least one of the knowledge base and the evidence-based medicine intelligent agent.

[0023] Evidence-based medicine intelligent agents: Medical AI systems that retrieve, analyze, annotate evidence, and conduct reverse questioning on clinical issues based on medical literature, clinical guidelines, knowledge graphs, and large model capabilities. Examples include Zheng Yuanfang, MedClaw, or similar services.

[0024] The aforementioned evidence task package can be a structured request generated from the smart glasses bedside session context, containing all or part of the following fields: session_id, turn_scope, known_fields, missing_fields, return_schema, and privacy_level.

[0025] Step S130 involves parsing and validating the structured result returned by the task processing object to obtain parsing and validation results. Parsing here refers to parsing the content fields, and validation refers to performing one or more checks on the result, such as identity, information security, routing, and validity.

[0026] Optionally, the structured results are parsed into suggestion objects, evidence objects, applicable condition objects, information gap objects, and risk warning objects according to a standard data structure. Conflict checks are performed on the structured results to obtain the results. Conflict checks include comparing the applicable conditions with the current patient's patient information and comparing the routing path of the structured results with the routing of the evidence task package (for example, if the evidence task package is routed locally, but the corresponding structured results come from a remote route, it indicates non-compliance; if the evidence task package uses remote route A, but the corresponding structured results come from remote route B, it indicates non-compliance; only when the two routes are consistent can it be considered compliant).

[0027] Step S140: Based on the verification result, display a prompt message on the smart glasses. This prompt message may be evidence to be confirmed or modified, or it may be a follow-up question or sentence segmentation required due to incomplete information.

[0028] In an optional embodiment, if the verification result shows that the conflict verification is passed, a small screen prompt message is generated for display on the smart glasses; in response to the doctor's confirmation or modification operation based on the small screen prompt message, a doctor feedback record is generated, and the doctor feedback record is associated with the evidence retrieval process of this diagnosis and treatment and stored in the audit log to form diagnosis and treatment evidence.

[0029] In another optional embodiment, if the verification result shows that the conflict verification has passed, and if there are information gap objects in the structured result, the information gap objects are mapped to bedside follow-up questions. If there are multiple bedside follow-up questions, a priority identifier is generated for each bedside follow-up question. The bedside follow-up questions are displayed on the smart glasses, specifically according to the priority identifier and the display limiting rules of the smart glasses, only displaying the highest priority bedside follow-up question at any given time. Reverse follow-up questions refer to supplementary consultation questions or examination items proactively proposed by the system when the information required for evidence reasoning is insufficient.

[0030] Through the above steps, when doctors wear smart glasses for bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is transformed into structured bedside context objects; the anonymized bedside context objects are constructed into evidence task packages and sent to the task processing object; the structured results returned by the task processing object are parsed and verified to obtain parsing results and verification results; based on the verification results, prompt information (such as search results, supplementary bedside follow-up questions, etc.) is displayed on the smart glasses in real time, which can solve the technical problem of low efficiency in accessing medical information through computers in related technologies.

[0031] This invention relates to a closed-loop evidence method for bedside diagnosis and treatment based on the collaboration of smart glasses and evidence-based medicine intelligent agents. When doctors wear smart glasses for rounds or consultations, the method abstracts the doctor-patient dialogue, bedside images, vital signs, and doctor confirmation information collected by the glasses into a desensitized clinical context, and converts it into an evidence task package that can be processed by an external evidence-based medicine intelligent agent. This external evidence-based medicine intelligent agent can be Zheng Yuanfang, MedClaw / Medical Intelligent Library (Skill Store), or similar evidence-based medicine intelligent agents. The system performs structured parsing, conflict verification, and small-screen limited display of the guidelines, literature, examination suggestions, medication risks, and reverse inquiry information returned by these agents, thereby forming an evidence closed loop of "bedside collection—evidence retrieval—reverse inquiry—doctor confirmation—audit archiving" (evidence closed loop: i.e., a continuous closed loop of smart glasses bedside collection, context abstraction, privacy desensitization, evidence task routing, external result parsing, security verification, reverse inquiry, doctor confirmation, and audit backfilling).

[0032] As an optional implementation, this solution utilizes smart glasses to collect doctor-patient dialogues, bedside images, and doctor operations, and displays prompts on the glasses. 1) In this solution, the voice, images, vital signs, and doctor confirmation information collected by the glasses are not limited to local prompts or medical record drafts, but can be sent to structured evidence task packages of external evidence-based medicine intelligent agents; 2) Bedside real-time conversations are characterized by short rounds, multiple field sources, and limited doctor attention. This solution can bind session_id, turn_id, source_ref, and doctor_status to external evidence results; 3) External evidence results can be stably backfilled to the current consultation round, prompted on the glasses, confirmed and recorded by the doctor, and the source of evidence can be restored in subsequent audits; 4) External intelligent agent suggestions can be used for secondary conflict isolation with local patient age, allergy history, pregnancy status, liver and kidney function, in-hospital pathways, and medical insurance rules.

[0033] See Figure 2The system used to implement the technical solution of this application may include a smart glasses terminal, a mobile terminal or edge gateway, a medical auxiliary service terminal, a session context management module, a privacy desensitization module, an evidence task orchestration module, an evidence-based medicine intelligent agent adapter, an evidence result parsing module, a conflict and risk verification module, a reverse questioning mapping module, a small screen prompt generation module, a doctor review and feedback module, and an audit storage module.

[0034] In practice, the smart glasses output four types of events: audio_event, image_event, vital_event, and doctor_action. The edge gateway writes these events into bedside_stream={session_id, event_id, source_type, source_ref, event_time, payload_hash, doctor_status}. The medical assistance server only receives the de-identified or structured bedside_stream and does not directly send the complete original audio and video to the external evidence-based medicine intelligent agent.

[0035] 1) Bedside conversation context abstraction See Figure 3 The system generates a session_id when the doctor begins rounds or consultations, and generates a turn_id or event_id for each round of patient responses, doctor confirmations, image acquisitions, and vital sign data. The session context management module converts raw audio transcription, translated text, chief complaints, symptoms, vital signs, examination results, past medical history, allergy history, image findings, and doctor confirmation status into structured context fields.

[0036] In practice, the session context is represented as bedside_context={session_id, patient_scope, turn_list[], field_list[], source_refs[], doctor_confirmations[], locked_fields[]}. Each field in field_list stores field_name, field_value, source_ref, turn_id, confidence, and confirm_state; locked_fields includes chief complaint, confirmed allergy history, key signs, and doctor-locked fields, which cannot be automatically pruned or overwritten by external results.

[0037] In the above smart glasses bedside conversation evidence task package protocol: audio_event, image_event, vital_event, and doctor_action on the smart glasses are converted into bedside_context, and evidence_task is constructed based on session_id, turn_scope, known_fields, missing_fields, privacy_level, and return_schema.

[0038] 2) Privacy classification and minimum necessary query construction The privacy desensitization module classifies context fields according to their sensitivity: P0 are fields that should not be shared externally, such as name, ID number, medical insurance number, contact information, and detailed address; P1 are replaceable fields, such as age can be converted into age range, gender can be retained as a clinically necessary attribute, and specific dates can be converted into relative disease course; P2 are clinically necessary fields, such as chief complaint, symptoms, signs, and abnormal examination results; P3 are fields that can be publicly disclosed, such as department, task type, and scope of evidence.

[0039] In practice, the system generates evidence_task={task_id, session_id, turn_scope, task_type, dept, privacy_level, known_fields, missing_fields, return_schema, timeout_ms}. The known_fields only retain the anonymized chief complaint, symptoms, signs, abnormal examination results, and doctor-confirmed fields; the P0 identity field is not written to the task package.

[0040] The evidence task orchestration module selects the minimum necessary fields from the bedside_context based on the task_type and generates field_filter_log={task_id, included_fields[], excluded_fields[], exclude_reason[]}; exclude_reason is used to record the reason why a field is excluded, such as a strong identity field, a low confidence field, an unconfirmed field, or a field that is irrelevant to the current task.

[0041] In the aforementioned privacy classification, field filtering, and evidence-based task routing mechanism: fields are classified into P0-P3 levels, a field_filter_log is generated, and local_only, local_first, external_allowed, or external_blocked routes are selected according to task_type, evidence_density, privacy_level, network_state, and local_hit_score.

[0042] 3) Evidence-based medicine intelligent agent task routing Task types include, but are not limited to, diagnostic direction support, examination recommendations, medication risk assessment, guideline comparison, literature summaries, medical insurance compliance verification, research question expansion, and patient education material generation. The system prioritizes querying local hospital paths and high-frequency knowledge bases; when local searches are insufficient, the task requires higher evidence density, or doctors actively trigger in-depth evidence-based medicine, external evidence-based medicine intelligent agents are invoked through adapters.

[0043] In practice, task routing is executed according to route=f(task_type, evidence_density, privacy_level, network_state, local_hit_score). The route value includes local_only, local_first, external_allowed, and external_blocked. When privacy_level exceeds the external threshold or network_state is unavailable, the task is restricted to local_only or local_first. External evidence-based medicine intelligent agents such as Zheng Yuanfang are invoked only when local_hit_score is below the threshold and privacy_level allows it.

[0044] The evidence-based medicine intelligent agent adapter unifies the return of different external services with the constraint return_schema={suggestion[], evidence[], condition[], gap[], risk[], source_meta[]}; if the external result cannot be parsed into this structure, the system marks it as unstructured_result and enters manual review, instead of directly prompting the glasses.

[0045] 4) Evidence analysis, conflict verification and isolation See Figure 4The evidence result parsing module breaks down the natural language or structured results returned by the external agent into suggestion objects, evidence objects, applicable condition objects, information gap objects, and risk objects. Evidence objects include at least source_type, source_title, guideline_clause, publication_time, evidence_level, quote_or_summary, and url_or_locator.

[0046] In practice, the externally returned content is parsed as result_object={task_id, suggestion[],evidence[],condition[],gap[],risk[]}. The system compares the patient's age, allergy history, pregnancy status, liver and kidney function, and in-hospital pathways item by item; suggestions that do not meet the condition are marked as isolated and do not enter the default prompt in the glasses.

[0047] The conflict check result is written to safety_check={task_id, suggestion_id, matched_fields[], conflict_type, safety_state, reviewer_required}. safety_state includes pass, isolated, need_confirm, and discard; only pass or need_confirm suggestions confirmed by the doctor can proceed to the next step in the case draft or on the glasses.

[0048] Since external evidence-based medicine intelligent agents such as Zheng Yuanfang return suggestions based on desensitized contexts, which cannot directly replace doctors' diagnostic judgments, this embodiment first breaks down the returned content into suggestion, evidence, condition, gap, and risk objects, and then performs conflict verification with the local patient context; its function is to limit external evidence to auxiliary information that is interpretable, isolable, and can be confirmed by doctors.

[0049] In the external evidence object parsing and local security isolation mechanism, external returns can be uniformly parsed into suggestion, evidence, condition, gap, and risk objects, and matched with local patient context, in-hospital path, and medical insurance rules through safety_check. Conflicts are suggested to enter the isolated or need_confirm state.

[0050] 5) Reverse questioning mapping and small screen limiting prompts See Figure 5 When the external evidence-based medicine intelligent agent returns insufficient information or asks follow-up questions, the follow-up question mapping module converts it into short sentences that can be directly asked at the bedside and sorts them according to the priority of danger signals, strength of evidence, questions already asked in the current session, and display frequency on the glasses.

[0051] In practice, the gap object is converted to prompt={prompt_id, task_id, question_text, priority, max_display_chars, confirm_required}. On the glasses device, only the highest priority item is displayed according to priority; other prompts are directed to the mobile or system device lists.

[0052] The small screen prompt generation module enforces a limit rule on each prompt: display_text must not exceed max_display_chars, and only one follow-up question, a maximum of three examination suggestions, and a small number of risk badges are allowed at the same time; if the doctor is performing a physical examination, operating, or speaking, low-priority prompts enter the waiting queue, and high-risk prompts can override the current prompt.

[0053] 6) Optimization of audit archiving and feedback The audit storage module records task_id, session_id, call time, name of the external agent called, pattern or Skill identifier, input field summary, privacy level, source of returned evidence, conflict verification results, and doctor's actions. Doctor feedback is used to adjust subsequent task routing, evidence density, follow-up question order, and department templates.

[0054] In practice, the audit record is saved as audit_record={audit_id, task_id, session_id, turn_scope, input_hash, route_state, privacy_level, external_agent, source_meta[], safety_state, doctor_action, output_prompt_id, timestamp}. This record is not a case evidence node in the general evidence chain, but is used to reconstruct the closed loop of a single bedside evidence call for smart glasses, from collection, desensitization, routing, return, verification to doctor processing.

[0055] Doctor feedback is written to feedback={doctor_id, task_id, suggestion_id, action_type, edit_text, reason_tag}, where action_type includes accepted, modified, ignored, and inapplicable. The system statistically analyzes feedback by department and task type, and updates the prompt sorting weight, default value of evidence density, and external call trigger threshold.

[0056] In the reverse inquiry small screen mapping and bedside call auditing mechanism, the gap object can be mapped to the prompt and displayed according to the glasses-side limiting rules. At the same time, the closed loop of collection, desensitization, routing, evidence source, security status, doctor operation and output prompts can be recorded through audit_record.

[0057] The technical solution adopted in this application: 1) Connects the real-time bedside conversations of doctors wearing smart glasses with the deep evidence capabilities of evidence-based medicine intelligent agents such as Zheng Yuanfang, reducing the need for doctors to manually transcribe patient information and switch terminals. 2) Through bedside_context, evidence_task, return_schema, and audit_record, the bedside conversations of smart glasses are stably bound to session_id, turn_id, and doctor_status. 3) Through P0-P3 privacy classification, minimum necessary field filtering, and local priority routing, the risk of sending patient privacy information to external intelligent agents is reduced. 4) Through evidence object parsing, applicable condition matching, safety_check, and isolated status, external AI suggestions are prevented from being displayed or archived without verification of the local patient context. 5) Through gap-to-prompt mapping and glasses-side limited display, the information gaps raised by the evidence-based intelligent agent are transformed into short, questionable sentences at the bedside in real time, forming a closed loop of bedside evidence.

[0058] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0059] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0060] According to another aspect of the embodiments of this application, a smart glasses-based diagnostic information interaction device is also provided for implementing the above-described smart glasses-based diagnostic information interaction method. Figure 6 This is a schematic diagram of an optional smart glasses-based diagnostic information interaction device according to an embodiment of this application, such as... Figure 6 As shown, the device may include: The conversion unit 61 is used to convert the multimodal bedside data collected by the smart glasses into a structured bedside context object when the doctor wears smart glasses to conduct bedside diagnosis and treatment. Optionally, the conversion unit is further configured to: generate a session identifier uniquely corresponding to the current medical session, and generate a round identifier for each round of doctor-patient interaction; convert the target events in the multimodal bedside data into structured fields in the bedside context object, and associate each structured field with its source, round identifier, confidence level, and confirmation status, wherein the target events include at least one of voice events, image events, vital sign events, and doctor action events; and set a locked field list in the bedside context object, wherein the fields in the locked field list are prohibited from being automatically pruned or overwritten during subsequent desensitization and task construction processes.

[0061] Sending unit 62 is used to construct the desensitized bedside context object into an evidence task package and send it to the task processing object; Optionally, the sending unit is further configured to: classify the fields in the bedside context object according to preset privacy classification rules, wherein the privacy classification includes identity fields that are prohibited from being sent out, replaceable indirect identification fields, and clinically necessary fields; generate the evidence task package using the clinically necessary fields and the transformed replaceable indirect identification fields in the bedside context object, and record the excluded fields and their exclusion reasons, generating a field filtering log; and send the evidence task package to the task processing object according to a routing strategy, wherein the routing strategy is selected from local routing priority, local routing only, external routing allowed, and external routing prohibited based on the task type, local knowledge base hit score, privacy level, and network status, and the task processing object includes at least one of a knowledge base and an evidence-based medicine agent.

[0062] The parsing and verification unit 63 is used to parse and verify the structured result returned by the task processing object to obtain the parsing result and the verification result; Optionally, the parsing and verification unit is further configured to: parse the structured result into suggestion objects, evidence objects, applicable condition objects, information gap objects, and risk warning objects according to a standard data structure; perform conflict verification on the structured result to obtain the verification result, wherein the conflict verification includes comparing the applicable conditions with the current patient's patient information and comparing the routing path of the structured result with the routing of the evidence task package.

[0063] The display unit 64 is used to display prompt information on the smart glasses based on the verification result.

[0064] Optionally, the display unit is further configured to: generate a small screen prompt message for display on the smart glasses if the verification result shows that the conflict verification has passed; generate a doctor feedback record in response to the doctor's confirmation or modification operation based on the small screen prompt message, and associate the doctor feedback record with the evidence retrieval process of this diagnosis and treatment and store it in the audit log to form diagnosis and treatment evidence.

[0065] Optionally, the display unit is further configured to: if the verification result shows that the conflict verification has passed, and if there is an information gap object in the structured result, map the information gap object to a bedside follow-up question; and display the bedside follow-up question on the smart glasses.

[0066] Optionally, the display unit is further configured to: map the information gap object to bedside follow-up questions, including: mapping the information gap object to multiple bedside follow-up questions and generating a priority identifier for each bedside follow-up question; and display the bedside follow-up questions on the smart glasses, including: displaying only the highest priority bedside follow-up question at any given time according to the priority identifier and the display limiting rules of the smart glasses.

[0067] Through the aforementioned modules, when doctors wear smart glasses for bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is transformed into structured bedside context objects; the anonymized bedside context objects are constructed into evidence task packages and sent to a task processing object; the structured results returned by the task processing object are parsed and verified to obtain parsing results and verification results; based on the verification results, prompt information (such as search results, supplementary bedside follow-up questions, etc.) is displayed on the smart glasses in real time, which can solve the technical problem of low efficiency in accessing medical information via computer in related technologies.

[0068] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in a corresponding hardware environment, and can be implemented through software or hardware, wherein the hardware environment includes a network environment.

[0069] According to another aspect of the embodiments of this application, a server or terminal for implementing the above-described method for interactive medical information based on smart glasses is also provided.

[0070] Figure 7 This is a structural block diagram of a terminal according to an embodiment of this application, such as... Figure 7 As shown, the terminal may include: one or more (only one is shown in the figure) processors 701, memory 703, and transmission devices 705, such as... Figure 7 As shown, the terminal may also include input / output devices 707.

[0071] The memory 703 can be used to store software programs and modules, such as the program instructions / modules corresponding to the smart glasses-based diagnostic information interaction method and device in this embodiment. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 703, thereby realizing the aforementioned smart glasses-based diagnostic information interaction method. The memory 703 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 703 may further include memory remotely located relative to the processor 701, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The aforementioned transmission device 705 is used to receive or send data via a network, and can also be used for data transfer between the processor and memory. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 705 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 705 is a radio frequency (RF) module used for wireless communication with the Internet.

[0073] Specifically, memory 703 is used to store application programs.

[0074] The processor 701 can invoke the application program stored in the memory 703 via the transmission device 705 to perform the following steps: When a doctor wears smart glasses to conduct bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is converted into a structured bedside context object; the anonymized bedside context object is constructed into an evidence task package and sent to a task processing object; the structured result returned by the task processing object is parsed and verified to obtain the parsing result and the verification result; based on the verification result, a prompt message is displayed on the smart glasses.

[0075] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0076] Those skilled in the art will understand that Figure 7The structure shown is for illustrative purposes only. The terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal devices. Figure 7 This does not limit the structure of the aforementioned electronic device. For example, the terminal may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 7 The different configurations shown.

[0077] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0078] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to execute program code for a diagnostic information interaction method based on smart glasses.

[0079] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0080] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: When a doctor wears smart glasses to conduct bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is converted into a structured bedside context object; the anonymized bedside context object is constructed into an evidence task package and sent to a task processing object; the structured result returned by the task processing object is parsed and verified to obtain the parsing result and the verification result; based on the verification result, a prompt message is displayed on the smart glasses.

[0081] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0082] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0083] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0084] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0085] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for interactive medical information based on smart glasses, characterized in that, include: When doctors wear smart glasses to conduct bedside diagnosis and treatment, the multimodal bedside data collected by the smart glasses is transformed into a structured bedside context object; The de-identified bedside context object is constructed into an evidence task package and sent to the task processing object; The structured results returned by the task processing object are parsed and verified to obtain the parsing results and the verification results; Based on the verification result, a prompt message is displayed on the smart glasses.

2. The method according to claim 1, characterized in that, The multimodal bedside data collected by the smart glasses is transformed into a structured bedside context object, including: Generate a session identifier that uniquely corresponds to the current medical session, and generate a round identifier for each round of doctor-patient interaction; The target events in the multimodal bedside data are converted into structured fields in the bedside context object, and each structured field is associated with its source, round identifier, confidence level, and confirmation status. The target events include at least one of voice events, image events, vital sign events, and doctor action events. Set a list of locked fields in the bedside context object. Fields in the locked field list are prohibited from being automatically clipped or overwritten during subsequent desensitization and task construction.

3. The method according to claim 1, characterized in that, The de-identified bedside context object is constructed into an evidence task package and sent to the task processing object, including: According to the preset privacy classification rules, the fields in the bedside context object are classified, and the privacy classification includes identity fields that are prohibited from being sent out, replaceable indirect identification fields, and clinically necessary fields. The evidence task package is generated using the clinically necessary fields and the transformed replaceable indirect identifier fields in the bedside context object, and the excluded fields and their reasons for exclusion are recorded to generate a field filtering log. The evidence task package is sent to the task processing object according to the routing policy. The routing policy is selected from local routing priority, local routing only, allowing external routing, and prohibiting external routing based on task type, local knowledge base hit score, privacy level, and network status. The task processing object includes at least one of knowledge base and evidence-based medicine intelligent agent.

4. The method according to claim 1, characterized in that, The structured results returned by the task processing object are parsed and validated to obtain parsing results and validation results, including: The structured results are parsed into suggestion objects, evidence objects, applicable condition objects, information gap objects, and risk warning objects according to the standard data structure; The structured results are subjected to conflict verification to obtain the verification result. The conflict verification includes comparing the applicable conditions with the current patient's patient information and comparing the routing path of the structured results with the routing of the evidence task package.

5. The method according to any one of claims 1 to 4, characterized in that, Based on the verification result, a prompt message is displayed on the smart glasses, including: If the verification result shows that the conflict verification has passed, a small screen prompt message is generated for display on the smart glasses; In response to the doctor's confirmation or modification operation based on the small screen prompt information, a doctor feedback record is generated, and the doctor feedback record is associated with the evidence retrieval process of this diagnosis and treatment and stored in the audit log to form diagnosis and treatment evidence.

6. The method according to any one of claims 1 to 4, characterized in that, Based on the verification result, a prompt message is displayed on the smart glasses, including: If the verification result shows that the conflict verification is passed, and if there is an information gap object in the structured result, the information gap object will be mapped to a bedside follow-up question. The bedside follow-up questions are displayed on the smart glasses.

7. The method according to claim 6, characterized in that, The bedside follow-up questions are displayed on the smart glasses, including: Mapping the information gap object to bedside follow-up questions includes: mapping the information gap object to multiple bedside follow-up questions, and generating a priority identifier for each bedside follow-up question; Displaying the bedside follow-up questions on the smart glasses includes: according to the priority identifier and the display limiting rules of the smart glasses, only the highest priority bedside follow-up question is displayed at any given time.

8. A diagnostic information interaction device based on smart glasses, characterized in that, include: The conversion unit is used to convert the multimodal bedside data collected by the smart glasses into a structured bedside context object when the doctor wears smart glasses to conduct bedside diagnosis and treatment. The sending unit is used to construct the de-identified bedside context object into an evidence task package and send it to the task processing object. The parsing and verification unit is used to parse and verify the structured results returned by the task processing object to obtain parsing results and verification results; The display unit is used to display prompt information on the smart glasses based on the verification result.

9. A computer-readable storage medium, characterized in that, The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when it is run.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the method described in any one of claims 1 to 7 through the computer program.

Images