Medical self-service machine multi-mode interaction guiding method and device, equipment and medium

Abstract

The invention relates to a medical self-service machine multi-mode interaction guiding method and device, equipment and a medium. The method comprises the following steps: acquiring a symptom description and / or a symptom image; performing standardization processing on the symptom description and / or the symptom image to obtain standardized symptom information; based on the standardized symptom information, knowledge enhancement reasoning is carried out, and a doctor seeing department list is obtained; based on the doctor-seeing department list, generating operation guidance for the doctor-seeing department; generating a pre-inquiry question based on the registration information; performing information extraction on the illness state description to obtain a medical history record; and sending the medical history record to a corresponding doctor terminal. The method can improve the doctor seeing efficiency and the service stability, and meets the diversified requirements of patients.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of intelligent medical guidance, and particularly relates to a medical self-service machine multi-modal interaction guidance method, device, equipment and medium. BACKGROUND

[0002] In the traditional medical guidance mode, hospitals mainly rely on manual guidance desks, fixed sign systems and simple electronic query devices to handle the guidance needs of patients. Manual guidance requires patients to queue up for consultation, and staff provide department suggestions according to memory or system query; the fixed sign system requires patients to recognize and understand complex floor plans by themselves; the electronic query device provides basic menu-based selection services, lacking intelligent interaction capabilities. The current traditional guidance mode has significant limitations: manual service is limited by the number of staff and professional level, and the response is slow and the quality is unstable during peak periods; the fixed sign system cannot provide personalized route planning, and patients often need to ask for directions multiple times; the simple electronic query device has a harsh interaction experience and is difficult to understand complex needs, especially for the elderly and patients with special needs, and cannot provide end-to-end full-process intelligent guidance services. SUMMARY

[0003] Therefore, it is necessary to provide a medical self-service machine multi-modal interaction guidance method, device, equipment and medium that can improve the efficiency of medical treatment, service stability and meet the diverse needs of patients.

[0004] In a first aspect, the application provides a medical self-service machine multi-modal interaction guidance method, comprising: obtaining a symptom description and / or a symptom image; and performing standardized processing on the symptom description and / or the symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom; performing knowledge-enhanced reasoning based on the standardized symptom information to obtain a list of treatment departments; generating operation instructions for the treatment departments based on the list of treatment departments; the operation instructions are used to instruct a user to perform registration operations on the treatment departments to generate registration information; generating pre-consultation questions based on the registration information; the pre-consultation questions are used to instruct a user to feed back a disease description; performing information extraction on the disease description to obtain a medical history record; and sending the medical history record to a corresponding doctor terminal.

[0005] Further, the knowledge-enhanced reasoning based on the standardized symptom information to obtain a list of treatment departments comprises: performing semantic analysis on the text content in the symptom information to obtain text features; transforming image data in the symptom information into a high-dimensional vector; and splicing the text features and the high-dimensional vector to obtain symptom feature data; traversing a preset knowledge base to associate the symptom feature data and medical entries in the knowledge base to obtain an initial candidate list; performing semantic correlation calculation on the associated entries in the initial candidate list; and sorting the medical entries based on the semantic correlation to obtain a knowledge point list; combining the knowledge point list and the symptom feature data to obtain context information; and inputting the context information into a large language model to obtain a clinic department list.

[0006] Further, the knowledge base is constructed by the following method: performing data cleaning on unstructured medical text data to obtain a medical term set; the medical text data includes at least one of digitized content of authoritative publications, public data of a medical knowledge platform, and a real medical record text after desensitization processing; identifying key medical entities from the medical term set based on a sequence labeling model; and identifying association relationships between the key medical entities; constructing a medical knowledge graph by taking the key medical entities as nodes and the association relationships as edges; the key medical entities include at least one of symptoms, diseases, and drugs; based on the confidence of the medical knowledge graph, evaluating the quality of the medical knowledge graph to obtain an evaluation result; based on the evaluation result, selecting an optimization scheme corresponding to the evaluation result to optimize the medical knowledge graph to obtain the knowledge base.

[0007] Further, information extraction is performed on the disease description to obtain a medical history record, including: based on a medical large model, performing deep semantic analysis on the disease description to obtain a preliminary list of medical entities; the medical entities include at least one of symptom entities, time expressions, degree modifiers, and property descriptions; based on a preset medical term library, mapping non-standardized descriptions in the preliminary list to standardized expressions to obtain a medical entity list; filling the medical entity list into corresponding fields of a preset case document framework to obtain an electronic medical record; the overall confidence of the electronic medical record is calculated by the following formula: ; wherein, is the overall confidence, , , is a weighting coefficient, is an entity recognition confidence, and n is the total number of entities, a knowledge base consistency score, a semantic completeness score; when the overall confidence is greater than a preset threshold, determining the electronic medical record as the medical history record.

[0008] Further, before generating the pre-consultation questions based on the registration information, the method further comprises: confirming an accurate location of a medical department corresponding to the registration information to obtain a destination location coordinate; calculating a current spatial location coordinate of the user based on the positioning beacon to obtain a real-time location coordinate; calculating a path between the destination location coordinate and the real-time location coordinate based on a path planning algorithm to obtain a navigation route; converting text data in the navigation route into voice data based on a TTS model; and converting the navigation route into a map guide based on a graphics rendering engine; the map guide is used to instruct a display device to generate a visualized travel route.

[0009] Further, the calculating of the current spatial location coordinate of the user based on the positioning beacon to obtain the real-time location coordinate comprises: sending a radio frequency positioning signal; the radio frequency positioning signal is used to instruct the positioning beacon to feed back original radio frequency signal data; calculating direction angle data of the radio frequency positioning signal to different positioning beacons based on the original radio frequency signal data; calculating a preliminary three-dimensional spatial coordinate of the user based on at least three direction angle data through a triangulation principle; filtering the preliminary three-dimensional spatial coordinate to obtain a high-precision spatial coordinate; mapping the high-precision spatial coordinate to a coordinate system of a hospital indoor map to obtain the real-time location beacon.

[0010] Further, the standardizing of the symptom description and / or the symptom image to obtain the standardized symptom information comprises: converting voice data in the symptom description into text based on a voice recognition engine; and performing encoding conversion on the text to obtain original text; performing stop word filtering on the original text based on natural language processing to obtain standard text; performing size standardization on the symptom image to obtain a standard image; and performing noise filtering on the standard image to obtain a denoised image; integrating the standard text and / or the denoised image to obtain the standardized symptom information.

[0011] In a second aspect, the present application further provides a medical self-service machine multi-modal interaction guiding device, comprising: a standardization module, configured to acquire a symptom description and / or a symptom image, and to perform standardization processing on the symptom description and / or the symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom; a reasoning module, configured to perform knowledge-enhanced reasoning based on the standardized symptom information to obtain a list of doctor's offices; a guidance module, configured to generate operation guidance for the doctor's offices based on the list of doctor's offices; the operation guidance is used to instruct a user to perform a registration operation on the doctor's offices to generate registration information; an inquiry module, configured to generate pre-inquiry questions based on the registration information; the pre-inquiry questions are used to instruct a user to feed back a disease description; a medical history module, configured to perform information extraction on the disease description to obtain a medical history record, and to send the medical history record to a corresponding doctor terminal.

[0012] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements any step of the method provided in the first aspect of the present application when executing the computer program.

[0013] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any step of the method provided in the first aspect of the present application.

[0014] The medical self-service machine multi-modal interaction guidance method, device, equipment and medium described above acquire a symptom description and / or a symptom image, and perform standardization processing on the symptom description and / or the symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom; knowledge-enhanced reasoning is performed based on the standardized symptom information to obtain a list of doctor's offices; operation guidance for the doctor's offices is generated based on the list of doctor's offices; the operation guidance is used to instruct a user to perform a registration operation on the doctor's offices to generate registration information; pre-inquiry questions are generated based on the registration information; the pre-inquiry questions are used to instruct a user to feed back a disease description; information extraction is performed on the disease description to obtain a medical history record; and the medical history record is sent to a corresponding doctor terminal. The method can understand complex requirements, improve support for the elderly and patients with special needs, provide end-to-end whole-process intelligent guidance services, improve doctor's office efficiency and service stability, and meet the diverse needs of patients. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to make the technical solutions in the embodiments of the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim to explain the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0016] Figure 1 A schematic diagram of a flow of a medical self-service machine multi-modal interaction guidance method is provided for an embodiment of the present application. Figure 2 A schematic diagram of the structure of a medical self-service machine multi-modal interaction guidance device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the technical solutions in the embodiments of the present application or the related art clearer, the accompanying drawings needed in the embodiments or the related art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim to explain the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0018] In one embodiment, as shown in Figure 1 A medical self-service machine multi-modal interaction guidance method is provided, and the embodiment takes the method applied to a terminal as an example, wherein the terminal can be a medical self-service machine. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction of the terminal and the server. In the embodiment, the method includes the following steps: Step 101, acquiring a symptom description and / or a symptom image; and performing standardized processing on the symptom description and / or the symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom.

[0019] The symptom description is a subjective statement provided by the user in the form of text or voice about their discomfort, containing key information such as symptom type, location, duration, and accompanying phenomena. The symptom image is visual material uploaded by the user related to the symptom, which may include, for example, photos of skin rashes, tongue coating images, or photos of injury sites, and must contain identifiable visual features to assist in analysis. The standardized symptom information is processed into a unified format after cleaning, conversion, and formatting. Text is converted into a text vector without redundant stop words and standardized medical terminology. Images are processed into a feature matrix with uniform size and noise reduction, serving as high-quality input for downstream analysis. If the input is voice, the terminal converts the audio to text through a speech recognition engine and processes dialects and noise interference. If the input is text, it is directly encoded and filtered for special characters. Natural language processing techniques are used to remove stop words and replace colloquial descriptions with standardized terminology. Images are standardized in size and filtered to reduce the interference of environmental light and obstructions. The processed text and image features are packaged into structured data objects, including original input, standardized content, and confidence scores.

[0020] Step 102, based on the standardized symptom information, knowledge-enhanced reasoning is performed to obtain a list of departments for consultation.

[0021] Specifically, knowledge-enhanced reasoning is a decision-making method that combines a medical knowledge base and a large language model. It retrieves authoritative medical knowledge to constrain and enhance the generation process of the large model, avoiding the illusion risk of pure model reasoning. The list of departments for consultation is a set of department names sorted by recommendation priority, with each department associated with a recommendation confidence and generated reasons. For example, the terminal extracts text features from the standardized symptom information through a semantic parsing model, generates high-dimensional vector image features through a neural network, and concatenates them into a multi-modal feature vector. The feature vector is used as a query condition to retrieve Top-50 (top 50) related medical items from the medical knowledge graph using the BM25 (Best Matching 25) algorithm. The Cross-Encoder model is used to calculate the semantic relevance of the query and the retrieval results, and the top five high-confidence knowledge fragments are selected. The knowledge fragments and symptom features are combined into a context prompt, which is input into the DeepSeek-R1 model through an API interface (Application Programming Interface) to generate a department recommendation list and explanation reasons.

[0022] Step 103, based on the list of departments for consultation, operation guidelines for the departments for consultation are generated; the operation guidelines are used to instruct the user to perform registration operations on the departments for consultation, and generate registration information.

[0023] Specifically, the operation guide is a set of multi-modal instructions for guiding the user to complete the registration operation, including visual highlight prompts, voice broadcasts, and dynamic arrow navigation. The registration information is structured data generated after the user completes the registration, containing fields such as department name, appointment time, patient ID, and cost. Optionally, the terminal calls the registration process description of the corresponding department from the template library according to the list of departments to be visited, detects the self-service machine interface elements using the YOLOv8 model, dynamically renders the highlight box and animated arrow, uses the TTS (Text To Speech) engine to convert the text guide to voice broadcast, and synchronously displays the visual prompts on the screen. After the user completes the registration operation, the input data is captured and structured registration information is generated.

[0024] Step 104, generating pre-diagnosis questions based on the registration information; the pre-diagnosis questions are used to indicate the user to feedback the disease description.

[0025] Specifically, the pre-diagnosis questions are an open or multiple-choice question set dynamically generated based on the registration department, used to collect disease details. The question design needs to comply with medical standards and be easy for patients to understand. The disease description is supplementary information provided by the user based on the pre-diagnosis questions, usually in the form of text or voice multi-turn conversation content. During the waiting period, the terminal calls the question template from the knowledge base and fills in the variables according to the characteristics of the registration department, controls the follow-up question logic through the finite state machine, converts the user's voice reply to text, and stores it in association with the question as a conversation log, to structure the disease in advance and reduce the doctor's inquiry time.

[0026] Step 105, information extraction is performed on the disease description to obtain a medical history record; and the medical history record is sent to a corresponding doctor terminal.

[0027] The disease description is a multi-turn conversation text provided by the user, containing unstructured descriptions such as symptoms, time, and degree. The medical history record is an electronic medical record fragment conforming to the FHIR (Fast Healthcare Interoperability Resources) standard, containing structured fields such as patient complaints, current medical history, and past medical history, which can be directly consulted by doctors through the doctor terminal. The doctor terminal is a terminal for directly displaying the medical history record. For example, the terminal uses the DeepSeek-7B model to identify medical entities in the text, maps the entities to standard codes through a medical terminology library, fills the entities into a pre-set medical record template, generates a preliminary electronic medical record, calculates the overall confidence of the comprehensive entity recognition accuracy, knowledge base consistency, and semantic completeness, and triggers manual review if the overall confidence is lower than a pre-set threshold, otherwise outputs the final medical history record to the corresponding doctor terminal, generates a high-quality, standardized medical history, reduces the doctor's paperwork burden, and ensures the reliability of clinical information.

[0028] The medical self-service machine multi-modal interaction guidance method provided by the embodiment comprises the following steps: acquiring symptom description and / or symptom image; performing standardized processing on the symptom description and / or symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom; performing knowledge-enhanced reasoning based on the standardized symptom information to obtain a clinic department list; generating operation instructions for the clinic department based on the clinic department list; the operation instructions are used to instruct a user to perform registration operation on the clinic department to generate registration information; generating pre-consultation questions based on the registration information; the pre-consultation questions are used to instruct the user to feed back a disease description; performing information extraction on the disease description to obtain a medical history record; and sending the medical history record to a corresponding doctor terminal. Through the above means, complex requirements can be understood, support for the elderly group and patients with special needs can be improved, end-to-end whole-process intelligent guidance services can be provided, and the efficiency of a clinic and the stability of a service can be improved to meet the diversified needs of patients.

[0029] In one of the embodiments, based on the standardized symptom information, knowledge-enhanced reasoning is performed to obtain a clinic department list, which comprises the following steps: Step 201: performing semantic analysis on the text content in the symptom information to obtain text features.

[0030] The text content in the symptom information is the text part in the standardized symptom information. The text has been cleaned and mapped to terms, and is a standardized medical description in a unified format. Semantic analysis is to understand the deep semantic structure of the text through a natural language processing model, including entity recognition, relationship extraction and semantic representation generation, and does not involve basic preprocessing, but focuses on extracting high-level semantic information. The text features are the numerical vector representation generated after analysis, which encodes the key semantic elements of the symptom and is used for downstream processing of a machine learning model. The terminal uses a pre-trained language model to encode the text. For example, the language model can be a medical field fine-tuned version of DeepSeek or BERT (Bidirectional Encoder Representations from Transformers), to generate high-dimensional semantic vectors, identify entities and their modification relationships, strengthen key information through an attention mechanism, suppress irrelevant descriptions, and output dense vectors as text features.

[0031] Step 202: converting the image data in the symptom information into a high-dimensional vector; and splicing the text features and the high-dimensional vector to obtain symptom feature data.

[0032] Specifically, the image data in the symptom information refers to the image part in the standardized symptom information, i.e., the denoised image that has completed size standardization and noise filtering, which is a clean image after preprocessing, rather than the original user-uploaded image. The high-dimensional vector is an image feature representation extracted by a convolutional neural network, which encodes deep visual features of the image. The symptom feature data is a multi-modal fusion result formed by splicing the text features and the image high-dimensional vector, which comprehensively represents the text and visual information of the symptom. The terminal uses a pre-trained visual model as a feature extractor. Exemplarily, the visual model can include a medical special version of YOLOv8 (You Only Look Once Version 8) or ResNet (Residual Network), which receives the standardized image, converts the image into a high-dimensional, numerical feature vector through its convolutional layers and fully connected layers. The model focuses on capturing visual patterns related to medical diagnosis, such as the morphology, edge features, and color uniformity of rashes.

[0033] In step 203, the symptom feature data and the medical entries in the preset knowledge base are associated to obtain an initial candidate list.

[0034] Specifically, the preset knowledge base is a structured database constructed based on a medical knowledge graph, which includes the association relationships among symptoms, diseases, and departments. The medical entries are standardized medical concepts in the knowledge base, accompanied by attributes and relationship edges. The initial candidate list is a preliminary matching result list generated after the association of the symptom feature data and the knowledge base entries, which includes possible related diseases, symptoms, and departments. The terminal takes the symptom feature data as a query vector, retrieves related medical entries in the knowledge base through similarity calculation, and encodes the knowledge base entities into vectors in advance. The BM25 retrieval algorithm is used to quickly match the symptom features and the knowledge base entries, and the preliminary associated candidate set is screened out. The matching scores of each candidate entry are calculated, and an initial list ranked in descending order of scores is generated, which can be accompanied by source evidence.

[0035] In step 204, the semantic relevance of the associated entries in the initial candidate list is calculated, and the medical entries are sorted based on the semantic relevance to obtain a knowledge point list.

[0036] The knowledge point list is a medical entry list sorted by relevance, which includes standardized terms and confidence levels, and is ranked from high to low in relevance. The terminal uses a cross-encoder model to calculate the fine-grained relevance of the symptom features and each candidate entry, and integrates the context information to be more accurate than the preliminary retrieval. The initial candidate list is reordered according to the relevance score, low-score entries are filtered out, and the top-5 high-confidence results are retained. The sorted entries and metadata are packaged into a structured list for use by the large model reasoning.

[0037] Step 205, combine the knowledge point list and symptom feature data to obtain context information; and input the context information into the large language model to obtain a list of consultation departments.

[0038] wherein the context information is a prompt combined from the knowledge point list and the symptom feature data, which contains standardized medical knowledge and user original symptom representation, and is used to guide the large model to generate reasonable output. The large language model is a generative model based on deep learning, has medical field knowledge, and can reason and generate natural language conclusions according to the context. The list of consultation departments is the recommended result output by the large model, with a generated reason. The terminal splices the high-relevance items in the knowledge point list and the symptom feature data into a structured prompt, inputs the context information into the large language model, and outputs a department recommendation list through generative decoding. The model generates an explanation based on a medical knowledge chain and logical reasoning.

[0039] The embodiment fuses the authority of the knowledge base and the reasoning ability of the large model to generate a reliable and explainable medical department recommendation, improves the trust of the user, and meets the diversified needs of the user.

[0040] In one of the embodiments, the knowledge base is constructed by the following method: Step 301, data cleaning is performed on unstructured medical text data to obtain a medical term set; the medical text data includes at least one of digitized content of authoritative publications, public data of a medical knowledge platform, and real medical record text after desensitization processing.

[0041] wherein the unstructured medical text data refers to original medical related text content without arrangement, including digitized versions of authoritative publications, public data of a medical knowledge platform, and real medical record text after desensitization processing. These data usually contain redundant information, non-standard terms, and noise. Exemplarily, the medical text data can include medical textbooks, clinical guidelines, encyclopedic websites, and medical records after removing personal information. The medical term set is a standardized medical vocabulary set extracted after cleaning, containing key terms such as symptoms, diseases, and drugs, each term conforming to medical standard naming for subsequent knowledge construction. The terminal encodes and unifies the original text, removes special characters and stop words using regular expressions, performs word segmentation processing to split long text into word or phrase units, replaces colloquial or dialect descriptions with standardized terms through a pre-defined medical term mapping table, verifies the correctness of the terms using a medical dictionary, identifies and merges duplicate terms using similarity calculation, filters out low-frequency or irrelevant terms, and finally outputs a pure and high-precision medical term set.

[0042] Step 302, based on a sequence labeling model, identify key medical entities from the medical term set; and identify the association relationship between the key medical entities.

[0043] Specifically, the sequence labeling model is a machine learning model for label prediction on text sequences, identifying entity boundaries and types. Key medical entities are core medical concepts identified from a medical term set, including symptoms, diseases, drugs, etc., each entity with a type label and context information. Association relationships are semantic connections between entities, such as symptom-disease relationships or drug-treatment relationships, which describe the internal logic of medical knowledge. The terminal uses the annotated medical text data to train the sequence labeling model, and the model learns to identify entity types and boundaries. Then infer the medical term set, output the entity label of each term, analyze the number of simultaneous occurrences of entities based on rules or deep learning models, identify relationship types, and store the identified entities and relationships as structured data, each entity with a confidence score, and the relationship with directionality and strength indicators.

[0044] Step 303: Constructing a medical knowledge graph with key medical entities as nodes and association relationships as edges; key medical entities include at least one of symptoms, diseases, and drugs.

[0045] Specifically, the medical knowledge graph is a graph-structured knowledge representation, with key medical entities as nodes and association relationships as edges, forming a networked knowledge model. The graph supports multi-hop queries and reasoning. Nodes represent medical entities, each node contains attributes such as entity name, type, source, and unique identifier. Edges represent relationships between entities, each edge contains relationship type, confidence, and evidence source. For example, the terminal defines the graph schema, including node types, relationship types, and attribute constraints, uses the Cypher language of the graph database to create the initial structure, imports entities as nodes into the graph database, assigns an ID (Identity, unique code) to each node, creates edges based on identified relationships, connects related nodes, sets edge attributes, creates indexes for commonly used query fields to speed up subsequent retrieval operations, and performs graph structure verification to ensure there are no isolated nodes or circular errors.

[0046] Step 304: Based on the confidence of the medical knowledge graph, evaluate the quality of the medical knowledge graph, and obtain the evaluation result.

[0047] wherein the confidence is a quantitative indicator representing the reliability score of nodes and relations in the knowledge graph, usually calculated based on data source authority, model output probability, and manual verification results. The evaluation result is a comprehensive score report on the overall quality of the knowledge graph, including accuracy, completeness, consistency, and other dimensions, used to guide optimization decisions. The terminal calculates the initial confidence for each node and edge, based on source weight, model confidence, and consistency check, using a weighted formula to aggregate the score. For example, the source weight can be defined as the score of authoritative publications; the model confidence can be defined as the output probability of the sequence labeling model; the consistency check can get a score by comparing with other knowledge bases. The quality evaluation dimensions can include accuracy, completeness, consistency, wherein accuracy is to verify whether the entities and relations are correct; completeness is to check whether key entities are covered comprehensively; consistency is to detect logical conflicts, generate evaluation reports, highlight weak links, output evaluation results, including overall confidence score, problem list and improvement suggestions.

[0048] Step 305, based on the evaluation result, selecting an optimization scheme corresponding to the evaluation result to optimize the medical knowledge graph, obtaining a knowledge base.

[0049] wherein the optimization scheme is an improvement strategy designed for the evaluation result, including data supplement, error correction, model retraining, etc., aiming to improve the quality of the knowledge graph. The knowledge base is the final knowledge storage system after optimization, containing the medical knowledge graph and its metadata, supporting efficient query and integration into application systems. The terminal selects the corresponding optimization scheme according to the evaluation result, performs incremental update, adds new nodes and edges, removes low-confidence elements, and recalculates the confidence. A version control system is used to manage changes, allowing rollback. The optimized knowledge graph is exported in a standard format with added metadata to obtain the knowledge base. For example, the optimization scheme includes: if the confidence is low, supplement authoritative data or retrain the model; if the coverage is not complete, crawl new data or expand the term set; if inconsistent, correct conflicting relations or introduce a rule engine.

[0050] This embodiment provides accurate and comprehensive medical knowledge support for intelligent diagnosis and reasoning by generating a high-quality, dynamically updateable knowledge base, improving the overall performance of the system.

[0051] In one of the embodiments, information extraction is performed on the disease description to obtain a medical history record, including: Step 401, based on the medical large model, performing deep semantic analysis on the disease description to obtain a preliminary list of medical entities; the medical entities include at least one of symptom entities, time expressions, degree modifiers, and property descriptions.

[0052] In particular, the medical large model is a large language model based on deep learning, which can be DeepSeek-7B optionally. The model is trained on a large amount of medical text and has the ability to understand and reason medical knowledge, and can analyze the semantic details in the condition description. The condition description is a free text form of symptom statement provided by the user, containing unstructured information such as symptoms, time, and degree. The preliminary list of medical entities is a set of unstandardized medical concepts extracted from the condition description, including: symptom entities for representing the patient's subjective discomfort; time expressions for describing the time of symptom occurrence or duration; degree modifiers for quantifying the frequency or intensity of symptoms; property descriptions for supplementing the characteristic attributes of symptoms. The terminal performs word segmentation and context encoding on the condition description through the medical large model, identifies entity boundaries and types through sequence labeling technology, assigns a confidence score to each identified entity to reflect the reliability of the model's identification, and outputs a preliminary list containing entity content, type label, and confidence, forming semi-structured data.

[0053] In step 402, the non-standardized description in the preliminary list is mapped to a standardized expression based on a preset medical terminology library to obtain a medical entity list.

[0054] The preset medical terminology library is a set of authoritative medical standard terms, including standardized expressions of symptoms, diseases, and drugs. The non-standardized description is a colloquial or ambiguous expression in the preliminary list that needs to be mapped to a standardized term. The medical entity list is a set of entities after standardization, and all descriptions are converted to standard expressions in the terminology library. The terminal uses a similarity algorithm to match the non-standardized description with the entries in the terminology library. For ambiguous terms, the most matching term is selected in combination with the context, and the mapped standard term is integrated with the information of the original entity through an artificial rule base to generate a pure medical entity list.

[0055] In step 403, the medical entity list is filled into the corresponding field of the preset case document framework to obtain an electronic medical record.

[0056] The preset case document framework is an electronic medical record template designed based on medical standards, containing structured fields for storing standardized medical information. The electronic medical record is a preliminary medical record document generated after filling the medical entity list into the template, containing structured data such as patient symptoms, time, and degree, but has not been quality verified. The terminal fills the content in the medical entity list into the corresponding field of the template according to the entity type. For example, symptom entities are filled into chief complaint or history of present illness; time expressions are filled into onset time; degree modifiers and property descriptions are attached as attributes of symptoms; the field integrity and logical reasonableness are checked; and a formatted electronic medical record is output, containing all filled fields and metadata.

[0057] In step 404, the overall confidence of the electronic medical record is calculated by the following formula: ; wherein, is the overall confidence, , , is the weighting coefficient, is the entity recognition confidence, n is the total number of entities, is the knowledge base consistency score, is the semantic completeness score.

[0058] Specifically, the overall confidence is a comprehensive score quantifying the reliability of the electronic medical record, ranging from 0 to 1, calculated by weighting the entity recognition accuracy, knowledge base consistency, and semantic completeness. The entity recognition confidence is the confidence score assigned by the medical large model for each entity, reflecting the accuracy of recognition. The knowledge base consistency score is the matching degree of the electronic medical record content with the medical knowledge base, calculated by comparing the entity relationships. The semantic completeness score is used to evaluate whether the electronic medical record covers the key elements of the disease, calculated based on the field filling rate and logical integrity. The terminal assigns weights according to clinical importance, takes the average of the entity recognition confidence, calculates the knowledge base consistency score through knowledge graph query matching relationship, and scores the semantic completeness based on template field filling rate. Substitute into the formula, output the overall confidence.

[0059] Step 405, when the overall confidence is greater than the preset threshold, determine the electronic medical record as the medical history record.

[0060] Specifically, the preset threshold is the qualified standard of the overall confidence, set by medical experts according to clinical needs, and below this value is considered to be insufficient in medical record quality. The medical history record is the final electronic medical record after confidence testing, which can be directly consulted by doctors or imported into the hospital information system for auxiliary diagnosis. The terminal compares the overall confidence with the preset threshold, if the overall confidence is greater than the threshold, adds the "verified" label to the electronic medical record, and stores it to the medical record database, and generates a printable or visual version; if the overall confidence is not greater than the threshold, record the failure reason and feedback to the previous step for optimization.

[0061] This embodiment realizes the function of pre-generating high-quality and reliable standardized medical history records in advance through pre-consultation, which can improve the efficiency of diagnosis and treatment, multi-dimensional quantitative evaluation, ensure the clinical reliability and usability of medical history records, and reduce the risk of medical errors.

[0062] In one of the embodiments, before generating the pre-consultation questions based on the registration information, it further includes: Step 501, confirm the accurate location of the medical department corresponding to the registration information, and obtain the destination location coordinates.

[0063] The registration information is structured data generated after the user completes the registration operation, and contains key information such as department name, number source time, floor number, room number, etc. The accurate location of the medical department is the physical location of the department in the hospital building, usually including three-dimensional space coordinates and floor plan description. The destination location coordinates are a digital representation of the department location, in the form of three-dimensional coordinate values, where the z-axis usually represents the floor height, used for precise positioning. The terminal parses the department name and floor information in the registration information, queries the hospital indoor map database, matches the standard location identifier of the department, and converts the logical location of the department into numerical coordinates in the physical coordinate system. Optionally, the coordinate system usually takes a fixed point in the hospital as the origin and uses the metric unit to output the destination location coordinates, including coordinate values, floor labels and auxiliary descriptions.

[0064] Step 502, based on the positioning beacon, calculate the current spatial position coordinates of the user, and obtain the real-time position coordinates.

[0065] Specifically, the positioning beacon is a hardware device deployed indoors in the hospital. Exemplarily, a Bluetooth AOA beacon can be used to provide a spatial reference point by emitting a radio frequency signal, and each beacon has a known fixed coordinate. The user's current spatial position coordinates are the real-time physical location of the terminal in the hospital, represented by three-dimensional coordinates and dynamically updated. The real-time position coordinates are high-precision position data after calculation and filtering, with a timestamp and a confidence score, used for navigation systems. The terminal receives radio frequency signals from multiple positioning beacons, extracts the original signal data, and calculates the user's preliminary three-dimensional coordinates based on the triangulation principle combined with the direction angle data of at least three beacons. A filtering algorithm is used to eliminate multipath effects and noise interference to improve accuracy. The filtered coordinates are mapped to the hospital's unified coordinate system to generate real-time position coordinates, which are updated regularly to provide reliable position input for dynamic navigation.

[0066] Step 503, based on the path planning algorithm, calculate the path between the destination location coordinates and the real-time position coordinates, and obtain the navigation route.

[0067] Specifically, the navigation route is the planned travel path, including path point sequence, turning instructions, distance estimation and expected time consumption. The terminal loads the hospital indoor map data, marks the feasible area, obstacles and key nodes, takes the real-time position coordinates as the starting point and the destination location coordinates as the end point, runs the path planning algorithm, avoids temporary obstacles in real time, optimizes the indicators, outputs the path point sequence, and adds semantic instructions. The route can support multi-floor switching and connection through elevators.

[0068] Step 504, based on the TTS model, convert the text data in the navigation route into voice data; and based on the graphics rendering engine, convert the navigation route into map guidance; the map guidance is used to instruct the display device to generate a visual travel route.

[0069] Wherein, the TTS model is a deep learning model that converts text instructions into natural speech, supporting multiple languages and dialects. The graphics rendering engine is a software component for generating visual maps, supporting dynamic rendering of paths, highlighting markers, and animation effects. The voice data is the audio stream of TTS output, containing navigation instructions. The map guide is a visual travel route, including a floor plan, path highlighting, direction arrows, and floor switching prompts. The terminal inputs the text instructions in the navigation route into the TTS model, generates voice data, and broadcasts it in real time through the device speaker. The graphics engine loads the hospital map base map, superimposes the navigation route, the user's current location, and key node markers. The voice instructions are synchronized with the visual elements in real time, and output through the display device.

[0070] This embodiment significantly reduces the user's operation burden by providing intuitive voice and visual dual-mode guidance. Through intelligent path planning, dynamic obstacle avoidance, and multi-floor path optimization, the travel time is shortened.

[0071] In one embodiment, based on the positioning beacon, the user's current spatial position coordinates are calculated to obtain real-time position coordinates, including: Step 601, sending a radio frequency positioning signal; the radio frequency positioning signal is used to indicate that the positioning beacon feeds back the original radio frequency signal data.

[0072] Wherein, the radio frequency positioning signal is a wireless electromagnetic wave signal actively emitted by the terminal, used to trigger the response of the surrounding positioning beacon. The signal contains device identifiers and timestamp information. The positioning beacon is a hardware device fixedly deployed indoors in the hospital, with a built-in antenna array and communication module, used to receive radio frequency signals and feed back data. The original radio frequency signal data is the original measurement data returned by the beacon after receiving the radio frequency signal, including signal strength, arrival timestamp, phase difference, and beacon coordinates. For example, the terminal emits a radio frequency signal at a specific frequency and power, all beacons within the signal coverage range receive it synchronously, and the transmission period is dynamically adjusted to balance power consumption and real-time performance. After receiving the signal, the positioning beacon records the signal parameters, including arrival time and phase information. The beacon encapsulates the data into a data packet, transmits it to the processor through the serial port, performs time synchronization and format unification, forms the original radio frequency signal data set, and feeds it back to the terminal.

[0073] Step 602, based on the original radio frequency signal data, calculate the direction angle data of the radio frequency positioning signal arriving at different positioning beacons.

[0074] Specifically, the direction angle data is the azimuth and elevation angle calculated from the phase difference of the radio frequency signal reaching the antenna array of different beacons, used to determine the direction vector of the signal source. The angle of arrival algorithm is the core algorithm for calculating the incident angle by analyzing the phase difference of the signal in the antenna array, which requires at least 2 antennas for one-dimensional positioning and 4 antennas for two-dimensional positioning. The terminal parses the phase difference sequence of the multi-antenna channel for each beacon returned by the terminal, uses the angle of arrival algorithm such as beamforming algorithm to calculate the signal incident angle, generates the direction vector, and the direction vector can be obtained by converting the phase difference into angle spectrum through fast Fourier transform, the peak value corresponds to the signal direction, and the angle data is bound with the beacon coordinates to output the direction angle data.

[0075] Step 603, based on at least three direction angle data, the preliminary three-dimensional space coordinates of the user are calculated by the triangulation principle.

[0076] Specifically, the triangulation principle is a positioning method based on the intersection of multiple sides in geometry, which uses at least three beacons with known coordinates and the measured direction angle to calculate the spatial coordinates of the terminal. The preliminary three-dimensional space coordinates are the user's position estimated by triangulation. The terminal selects the three and above beacons with the highest angle quality, where the highest angle quality can be equivalent to strong signal strength, high angle confidence, etc. The following equation set is established with the beacon coordinates as the known points and the direction angle as the constraints: ; Where x, y, and z are the preliminary three-dimensional space coordinates to be solved, 、 、 、 、 、 are the known beacon coordinates, 、 is the azimuth angle, is the elevation angle, and the over-determined equation set is solved by the least squares method to obtain the preliminary three-dimensional coordinates, and the obvious outliers are removed to output the preliminary coordinates.

[0077] Step 604, filtering the preliminary three-dimensional space coordinates to obtain high-precision space coordinates.

[0078] Where the high-precision space coordinates are the user's position coordinates after filtering and optimization, with motion state information such as speed and acceleration. The terminal establishes a user motion state model as the prediction basis of the filter, updates and iterates with the preliminary coordinates as the observation value and the motion model, predicts the current position according to the state of the last time, corrects the predicted value with the new observation value, reduces the noise influence, and outputs the high-precision coordinates after filtering, and calculates the confidence to ensure the reliability.

[0079] Step 605, map the high-precision spatial coordinates to the coordinate system of the hospital indoor map to obtain the real-time position beacon.

[0080] Wherein, the hospital indoor map coordinate system is a digital map coordinate system constructed based on the actual building drawings, with the origin being the building reference point and the unit being meters, containing floor layout, obstacles and key point annotations. The real-time position beacon is the final user position data mapped to the map coordinate system, used for real-time display and path planning of the navigation system. The terminal converts the high-precision spatial coordinates from the global coordinate system to the map local coordinate system through affine transformation, associates the floor information, checks whether the coordinates fall within the feasible area, projects along the nearest feasible point if they fall within the obstacle area, and outputs the real-time position beacon, including coordinates, timestamp, confidence and motion state.

[0081] This embodiment calculates the real-time position through triangulation, reduces the error of the user's position, seamlessly integrates the user's position with the hospital map, supports real-time navigation and dynamic path updating, and improves the accuracy of navigation.

[0082] In one embodiment, the symptom description and / or symptom image are standardized to obtain standardized symptom information, including: Step 701, based on the voice recognition engine, convert the voice data in the symptom description into text; and perform encoding conversion on the text to obtain the original text.

[0083] Wherein, the voice recognition engine is a deep learning-based audio processing system that can convert human speech into corresponding text content, usually including an acoustic model, a language model and a decoder, supporting recognition in various dialects and noisy environments. The voice data in the symptom description is the audio signal describing the symptoms input by the user through the recording device, which may contain dialects, colloquial expressions or environmental noise. The text is the preliminary text result output by the voice recognition engine and the symptom description text input by the user, which may contain recognition errors or non-standard expressions. Encoding conversion is the process of converting text from one character encoding format to a unified standard format, aiming to eliminate garbled codes and compatibility problems. The original text is the pure text content after encoding conversion, which retains the original output of voice recognition and the original text of symptom description, with a unified character encoding standard. The terminal receives audio input through the voice recognition engine, first performs preprocessing, then extracts features through the acoustic model, generates candidate text sequences through the language model, and finally outputs the most likely text result through the decoder, splices it with the symptom description text, detects the original encoding format of the recognized text, converts it to UTF-8 (Universal Character Set / Unicode Transformation Format, 8-bit) format to ensure correct display of special characters, and performs basic verification on the converted text to generate the original text for subsequent processing.

[0084] At step 702, the original text is filtered by stop words based on natural language processing to obtain standard text.

[0085] Specifically, natural language processing is a technology for processing and analyzing text, including word segmentation, entity recognition, syntax analysis, etc., for extracting and standardizing key information in the text. The standard text is the text after cleaning and standardization, only retaining key medical terms and descriptions, without redundant words and noise. Using natural language processing tools, the original text is split into a word sequence and tagged with part of speech. Based on a predefined stop word list, irrelevant words are filtered, common colloquial expressions are replaced based on a lightweight rule base, and the standard text is output.

[0086] At step 703, the symptom image is size-standardized to obtain a standard image, and the standard image is noise-filtered to obtain a denoised image.

[0087] Specifically, the symptom image is visual data uploaded by the user related to the symptom, which may have problems such as different sizes, uneven light, or noise interference. The standard image is an image with uniform size, which is convenient for model processing, but may still contain noise. The denoised image is a high-quality image after filtering, with clearer features and better feature extraction. The terminal uses an interpolation algorithm to scale the image to the target size, maintains the aspect ratio, analyzes the image noise type, selects an appropriate filter, for example, Gaussian filter smooths high-frequency noise; median filter eliminates outlier pixels, calculates the image signal-to-noise ratio to evaluate the denoising effect, and generates an optimized denoised image.

[0088] At step 704, the standard text and / or denoised image are integrated to obtain standardized symptom information.

[0089] The standardized symptom information is the final output structured data object, containing cleaned text features, image features, and metadata for downstream inference modules. The terminal pairs the standard text with the denoised image according to the timestamp or semantic association, packages the text features, image features, and metadata into a standard format, checks the data integrity, and outputs the standardized symptom information.

[0090] This embodiment converts voice data into text, pre-processes and encapsulates text data and image data, realizes seamless fusion of multi-modal data, provides high-quality and standardized input for knowledge-enhanced reasoning, and improves the accuracy of subsequent department recommendations.

[0091] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0092] Based on the same inventive concept, the embodiments of the present application also provide a medical self-service machine multi-modal interaction guidance device for implementing the medical self-service machine multi-modal interaction guidance method described above. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more medical self-service machine multi-modal interaction guidance device embodiments provided below can refer to the limitations of the medical self-service machine multi-modal interaction guidance method described above, which will not be repeated here.

[0093] In an exemplary embodiment, as shown in Figure 2 a medical self-service machine multi-modal interaction guidance device 800 is provided, comprising: a standardization module 801 configured to acquire a symptom description and / or a symptom image, and perform standardization processing on the symptom description and / or the symptom image to obtain standardized symptom information; the symptom description is a text uploaded by a user to describe a symptom; an inference module 802 configured to perform knowledge-enhanced inference based on the standardized symptom information to obtain a list of departments for treatment; a guidance module 803 configured to generate operation guidance for the department for treatment based on the list of departments for treatment; the operation guidance is used to instruct a user to perform a registration operation on the department for treatment to generate registration information; an inquiry module 804 configured to generate a pre-interrogation question based on the registration information; the pre-interrogation question is used to instruct a user to feed back a disease description; a medical history module 805 configured to perform information extraction on the disease description to obtain a medical history record, and send the medical history record to a corresponding doctor terminal.

[0094] Further, the inference module 802 is further configured to: perform semantic analysis on text content in the symptom information to obtain a text feature; convert image data in the symptom information into a high-dimensional vector, and concatenate the text feature and the high-dimensional vector to obtain symptom feature data; Traverse the preset knowledge base, associate the symptom feature data and the medical entries in the knowledge base, and obtain an initial candidate list; Perform semantic correlation calculation on the associated entries in the initial candidate list, and sort the medical entries based on the semantic correlation to obtain a knowledge point list; Combine the knowledge point list and the symptom feature data to obtain context information, and input the context information into a large language model to obtain a clinic department list.

[0095] Further, the reasoning module 802 is further configured to: Perform data cleaning on the unstructured medical text data to obtain a medical term set; the medical text data includes at least one of digitized content of authoritative publications, public data of a medical knowledge platform, and a real medical record text after desensitization processing; Identify key medical entities from the medical term set based on a sequence labeling model; and identify the association relationship between the key medical entities; Construct a medical knowledge graph with the key medical entities as nodes and the association relationship as edges; the key medical entities include at least one of symptoms, diseases, and drugs; Based on the confidence of the medical knowledge graph, evaluate the quality of the medical knowledge graph to obtain an evaluation result; Based on the evaluation result, select an optimization scheme corresponding to the evaluation result to optimize the medical knowledge graph to obtain a knowledge base.

[0096] Further, the medical history module 805 is further configured to: Based on the medical large model, perform deep semantic analysis on the disease description to obtain a preliminary list of medical entities; the medical entities include at least one of symptom entities, time expressions, degree modifiers, and property descriptions; Based on the preset medical term library, map the non-standardized description in the preliminary list to a standardized expression to obtain a medical entity list; Fill the medical entity list into the corresponding field of the preset case document framework to obtain an electronic medical record; Calculate the overall confidence of the electronic medical record by the following formula: ; Wherein, is the overall confidence, , , is a weighting coefficient, is an entity recognition confidence, n is the total number of entities, is a knowledge base consistency score, is a semantic completeness score; When the overall confidence is greater than a preset threshold, the electronic medical record is determined as the medical history record.

[0097] Further, the apparatus further comprises a navigation module, configured to: confirm an accurate position of a medical department corresponding to the registration information, to obtain a destination position coordinate; calculate a current spatial position coordinate of the user based on the positioning beacon, to obtain a real-time position coordinate; calculate a path between the destination position coordinate and the real-time position coordinate based on a path planning algorithm, to obtain a navigation route; convert text data in the navigation route into voice data based on a TTS model, and convert the navigation route into a map guide based on a graphics rendering engine; the map guide is used to instruct a display device to generate a visualized travel route.

[0098] Further, the navigation module is further configured to: send a radio frequency positioning signal; the radio frequency positioning signal is used to instruct the positioning beacon to feed back original radio frequency signal data; calculate direction angle data of the radio frequency positioning signal to different positioning beacons based on the original radio frequency signal data; calculate a preliminary three-dimensional spatial coordinate of the user through a triangulation principle based on at least three direction angle data; filter the preliminary three-dimensional spatial coordinate, to obtain a high-precision spatial coordinate; map the high-precision spatial coordinate to a coordinate system of a hospital indoor map, to obtain a real-time position beacon.

[0099] Further, the standardization module 801 is further configured to: convert voice data in the symptom description into text based on a voice recognition engine, and perform encoding conversion on the text, to obtain original text; perform stop word filtering on the original text based on natural language processing, to obtain standard text; perform size standardization on the symptom image, to obtain a standard image; and perform noise filtering on the standard image, to obtain a denoised image; integrate the standard text and / or the denoised image, to obtain standardized symptom information.

[0100] In an embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor implements steps of a medical self-service machine multi-modal interaction guidance method as described above when executing the computer program.

[0101] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement steps in each method embodiment described above.

[0102] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be seen from the part of the method embodiment. The device embodiment described above is only schematic, wherein the components shown as separate components can or can not be physically separate, and the components shown as a unit can or can not be physical units, i.e., can be located in one place or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement without creative labor.

[0103] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A multimodal interactive guidance method for a medical self-service machine, characterized in that, The method includes: Acquire symptom descriptions and / or symptom images; and perform standardization processing on the symptom descriptions and / or symptom images to obtain standardized symptom information; the symptom descriptions are text uploaded by users to describe the symptoms; Based on the standardized symptom information, knowledge-enhanced reasoning is performed to obtain a list of departments for consultation; Based on the list of medical departments, operation instructions are generated for each medical department; the operation instructions are used to instruct the user to register for an appointment at the medical department and generate registration information. Based on the registration information, a pre-consultation question is generated; the pre-consultation question is used to instruct the user to provide a description of their condition. Information is extracted from the description of the illness to obtain a medical history record; and the medical history record is sent to the corresponding doctor's terminal.

2. The method according to claim 1, characterized in that, Based on the standardized symptom information, knowledge-enhanced reasoning is performed to obtain a list of departments for consultation, including: Semantic analysis is performed on the text content of the symptom information to obtain text features; The image data in the symptom information is converted into a high-dimensional vector; and the text features and the high-dimensional vector are concatenated to obtain the symptom feature data. Traverse the preset knowledge base, associate the symptom feature data with the medical entries in the knowledge base, and obtain an initial candidate list; The semantic relevance of the related entries in the initial candidate list is calculated; and the medical entries are sorted based on the semantic relevance to obtain a knowledge point list. The knowledge point list and the symptom feature data are combined to obtain contextual information; the contextual information is then input into a large language model to obtain the list of departments to be visited.

3. The method according to claim 2, characterized in that, The knowledge base was constructed using the following methods: Unstructured medical text data is cleaned to obtain a set of medical terms; the medical text data includes at least one of the following: digitized content of authoritative publications, publicly available data from medical knowledge platforms, and anonymized authentic medical record texts; Based on the sequence labeling model, key medical entities are identified from the medical terminology set; and the relationships between the key medical entities are identified. A medical knowledge graph is constructed using the key medical entities as nodes and the relationships as edges. The key medical entities include at least one of symptoms, diseases, and drugs; Based on the confidence level of the medical knowledge graph, the quality of the medical knowledge graph is evaluated, and the evaluation result is obtained. Based on the evaluation results, an optimization scheme corresponding to the evaluation results is selected to optimize the medical knowledge graph, thereby obtaining the knowledge base.

4. The method according to claim 1, characterized in that, The process of extracting information from the description of the illness to obtain a medical history record includes: Based on a large medical model, deep semantic analysis is performed on the disease description to obtain a preliminary list of medical entities; the medical entities include at least one of the following: symptom entities, time expressions, degree modifiers, and property descriptions; Based on a pre-defined medical terminology database, the non-standardized descriptions in the preliminary list are mapped to standardized expressions to obtain a list of medical entities. The list of medical entities is populated into the corresponding fields of the preset medical record document framework to obtain an electronic medical record; The overall confidence level of the electronic medical record is calculated using the following formula: ； in, For the overall confidence level, , , These are weighting coefficients. The confidence level for entity identification is given by n, where n is the total number of entities. The knowledge base consistency score. The semantic completeness score; When the overall confidence level is greater than a preset threshold, the electronic medical record is identified as the medical history record.

5. The method according to any one of claims 1 to 4, characterized in that, Before generating pre-consultation questions based on the registration information, the process also includes: Confirm the exact location of the medical department corresponding to the registration information to obtain the destination location coordinates; Based on the positioning beacon, the user's current spatial coordinates are calculated to obtain the real-time location coordinates; Based on the path planning algorithm, the path between the destination location coordinates and the real-time location coordinates is calculated to obtain the navigation route; Based on the TTS model, the text data in the navigation route is converted into voice data; and based on the graphics rendering engine, the navigation route is converted into map guidance; the map guidance is used to instruct the display device to generate a visual travel route.

6. The method according to claim 5, characterized in that, The step of calculating the user's current spatial coordinates based on the positioning beacon to obtain the real-time location coordinates includes: Send a radio frequency positioning signal; the radio frequency positioning signal is used to instruct the positioning beacon to return raw radio frequency signal data; Based on the original radio frequency signal data, calculate the direction angle data of the radio frequency positioning signal reaching different positioning beacons; Based on at least three of the aforementioned directional angle data, the user's preliminary three-dimensional spatial coordinates are calculated using the principle of triangulation. The preliminary three-dimensional spatial coordinates are filtered to obtain high-precision spatial coordinates; The high-precision spatial coordinates are mapped onto the coordinate system of the hospital's indoor map to obtain the real-time location beacon.

7. The method according to claim 1, characterized in that, The standardization process for the symptom descriptions and / or symptom images to obtain standardized symptom information includes: Based on a speech recognition engine, the speech data in the symptom description is converted into text; and the text is encoded and converted to obtain the original text. Based on natural language processing, stop word filtering is performed on the original text to obtain standard text; The symptom images are sized and standardized to obtain standard images; then, noise is filtered from the standard images to obtain denoised images. The standardized symptom information is obtained by integrating the standard text and / or the denoised image.

8. A multimodal interactive guidance device for a medical self-service machine, characterized in that, The device includes: A standardization module is used to acquire symptom descriptions and / or symptom images; and to standardize the symptom descriptions and / or symptom images to obtain standardized symptom information; the symptom descriptions are text descriptions of symptoms uploaded by users; The reasoning module is used to perform knowledge-enhanced reasoning based on the standardized symptom information to obtain a list of departments to visit. The guidance module is used to generate operation guidance for the listed medical departments based on the list of medical departments; the operation guidance is used to instruct the user to register for the medical department and generate registration information. The consultation module is used to generate pre-consultation questions based on the registration information; the pre-consultation questions are used to instruct the user to provide a description of their condition. The medical history module is used to extract information from the description of the illness to obtain a medical history record; and to send the medical history record to the corresponding doctor's terminal.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images