Pre-inquiry method, pre-inquiry system, pre-inquiry equipment and electronic medium

Abstract

The invention discloses a pre-inquiry method, a pre-inquiry system, pre-inquiry equipment and an electronic medium. Obtaining pre-inquiry information provided by a user, wherein the pre-inquiry information comprises at least one of voice, text, picture and attachment; performing compliance auditing on the attachment to obtain an auditing result; generating an interaction problem based on the audit result and the pre-inquiry information, and performing multiple rounds of dialogues based on the interaction problem; and obtaining user answers through the multiple rounds of dialogues, and generating a structured pre-inquiry nodule according to the user answers. The method further comprises the steps of converting the voice into a text, obtaining key information by adopting a picture recognition technology, and processing an attachment by adopting a text large model to generate a structured text; dialogue is guided through a digital human, and dialogue content is dynamically adjusted according to user intentions and answers. According to the invention, intelligent processing of multi-modal information can be realized, the accuracy and efficiency of pre-inquiry are improved, and structured reference information is provided for subsequent medical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence in medicine, specifically to a pre-diagnosis method, system, device, and electronic medium. Background Technology

[0002] With the deepening application of artificial intelligence technology in the healthcare field, intelligent pre-diagnosis systems are playing an increasingly important role as a crucial bridge connecting patients and medical resources. By collecting basic patient information and symptom descriptions, pre-diagnosis systems provide a reference for subsequent formal diagnosis and treatment, effectively improving the efficiency of medical resource utilization and the accuracy of diagnosis and treatment.

[0003] However, traditional pre-consultation systems suffer from mechanical interaction and data management deficiencies: their rigid form-based interactions fail to build trust between doctors and patients, and static questionnaires cannot dynamically obtain comprehensive patient data; furthermore, the systems are rigid and lack departmental customization capabilities, and are devoid of medical-grade data review mechanisms, resulting in insufficient adaptability and significant compliance risks. Finally, their ability to generate pre-consultation summaries is limited. Most systems can only simply list question-and-answer records, lacking the ability to conduct in-depth analysis and structured summaries based on medical knowledge. This makes it difficult for pre-consultation results to directly provide valuable references for doctors, reducing the actual effectiveness of the pre-consultation process.

[0004] Therefore, there is an urgent need for a pre-diagnosis method that can provide a good interactive experience, accurately collect information, have flexible configuration capabilities, ensure data compliance, and generate high-quality structured summaries, so as to improve the efficiency and quality of pre-diagnosis. Summary of the Invention

[0005] To address the technical problems of existing pre-consultation systems, such as poor interactive experience, incomplete / inaccurate information collection, lack of flexibility and configurability, compliance and data quality risks, and limited ability to generate pre-consultation summaries, and to achieve technical effects such as improving the efficiency and quality of pre-consultation, optimizing user experience, enhancing the system's intelligence level, improving medical compliance and data availability, and providing high flexibility and scalability, this invention provides a pre-consultation method and system.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] Provide a pre-consultation method, including:

[0008] Obtain pre-consultation information provided by the user, wherein the pre-consultation information includes at least one of voice, text, images, and attachments;

[0009] The aforementioned attachments underwent a compliance review, and the review results were obtained.

[0010] Based on the review results and the pre-consultation information, interactive questions are generated, and multi-round dialogues are conducted based on the interactive questions.

[0011] User responses are obtained through the multi-round dialogue, and a structured pre-diagnosis summary is generated based on the user responses.

[0012] Preferably, obtaining the pre-consultation information provided by the user specifically includes:

[0013] For the aforementioned speech, speech recognition technology is used to obtain text information;

[0014] For the image, image recognition technology is used to obtain key information about the image;

[0015] For the attachments, a large text model is used to process them and generate structured text.

[0016] Preferably, the review results include at least one review of privacy information, format specifications, and medical content logic; the compliance review of the attachments, resulting in the following review results:

[0017] The privacy information in the attached text is detected using the detection model, and the privacy information review results are obtained.

[0018] The image clarity and file format compliance are judged according to preset requirements to obtain the format compliance review result;

[0019] The logic of the pre-consultation information and the attached content is judged according to preset rules to obtain the medical content logic review result.

[0020] Preferably, the step of generating interactive questions based on the review results and the pre-consultation information, and conducting multi-turn dialogues based on the interactive questions, includes:

[0021] Based on the user's selection, obtain the corresponding pre-consultation process;

[0022] Update the conversation status based on the audit results and user responses;

[0023] The redirection logic is triggered based on the updated dialogue state;

[0024] The process of generating follow-up questions based on knowledge graphs and user answers.

[0025] Preferably, the pre-consultation process further includes:

[0026] The attachment upload node is used to request the upload of medical documents related to the current symptoms;

[0027] Data acquisition nodes are used to obtain health information in preset categories.

[0028] Preferably, the multi-turn dialogue is performed by a digital human virtual assistant, and the process of obtaining user responses through multi-turn dialogue and generating a structured pre-diagnosis summary based on the dialogue results includes:

[0029] A digital human virtual assistant is determined based on the pre-diagnosis information;

[0030] Based on the user's response, obtain the user's intent;

[0031] The dialogue content is dynamically adjusted based on the user's intent and the user's response.

[0032] Preferably, determining the digital human virtual assistant based on the pre-consultation information includes the following steps: matching a corresponding 2D or 3D digital human image based on the user's personal information; allowing the user to adjust the digital human's appearance according to their preferences; and using the adjusted digital human as a virtual assistant.

[0033] Preferably, the dialogue content is dynamically adjusted based on the user's intent and the user's response, including:

[0034] Get the context information of the current conversation;

[0035] Obtain the analysis results of the attachments;

[0036] The analysis results and contextual information are used to infer the reasoning results;

[0037] Based on the reasoning results, the next round of questions is predicted using preset dialogue logic.

[0038] The present invention also provides a pre-diagnosis system, comprising:

[0039] The acquisition module is used to acquire pre-consultation information provided by the user, wherein the pre-consultation information includes at least one of voice, text, images and attachments;

[0040] The review module is used to review the attachments for compliance and obtain the review results;

[0041] The interaction module is used to generate interactive questions based on the review results and the pre-diagnosis information, and to conduct multi-round dialogues based on the interactive questions;

[0042] The generation module is used to obtain user answers through the multi-turn dialogue and generate a structured pre-diagnosis summary based on the user answers.

[0043] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the pre-diagnosis method as described above.

[0044] The present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the pre-diagnosis method as described above.

[0045] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the pre-diagnosis method as described above.

[0046] The beneficial effects of this invention are as follows: Through multimodal interaction, patients can provide information more conveniently; the intelligent follow-up questioning mechanism can delve deeper into the details of the patient's condition, and combined with attachment analysis, it avoids information omissions and repeated questions, thereby collecting more comprehensive and accurate patient information, saving doctors time in diagnosis and improving diagnostic accuracy; the customizable and vivid digital human makes the interaction process more natural and friendly, alleviating patients' tension when facing traditional cold interfaces, and supports multiple input methods, greatly facilitating users of different ages and with different usage habits; AI can grasp the contextual connections of multi-turn dialogues and intelligently integrate the content of attachments uploaded by users into the questioning logic, achieving truly "personalized" consultation; the automatically generated professional and structured pre-consultation summary can directly assist doctors in quickly grasping the patient's condition; the built-in attachment review function helps to discover and prompt the handling of potential privacy leakage risks, while ensuring the quality of uploaded medical materials, laying a good foundation for subsequent medical data analysis and application; rich configuration functions enable the system to flexibly adapt to the specific pre-consultation needs of different hospitals, departments, and diseases. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating a pre-consultation method in one embodiment;

[0048] Figure 2 This is a schematic diagram illustrating the interaction between multimodal input and attachment review in one embodiment;

[0049] Figure 3 This is a schematic diagram illustrating the working principle of multi-turn dialogue in an example.

[0050] Figure 4 This is a flowchart of a pre-consultation method in one embodiment;

[0051] Figure 5 This is a schematic diagram of a pre-diagnosis system provided in this manual;

[0052] Figure 6 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1

[0055] A pre-consultation method is proposed, which collects multimodal health data from users through various information channels and provides professional pre-consultation services after intelligent analysis. The specific implementation steps are as follows:

[0056] S100: Obtain pre-consultation information provided by the user, wherein the pre-consultation information includes at least one of voice, text, images, and attachments;

[0057] The system receives pre-consultation information from users through various channels. This information may include one or more of the following: voice, text, images, and attachments. The pre-consultation information provided by the user can be a voice response, or a voice response combined with a report text. Optionally, the pre-consultation information may also include the user's health data.

[0058] In one embodiment, pre-consultation information provided by the user is obtained, the pre-consultation information including at least one of voice, text, images, and attachments, specifically including:

[0059] S101 For the aforementioned speech, speech recognition technology is used to obtain text information;

[0060] S102 For the image, image recognition technology is used to obtain key information of the image;

[0061] S103 For the attachment, a large text model is used for processing to generate structured text;

[0062] Specifically, for voice information, speech recognition technology is used to convert it into text format to obtain text information; optionally, medical-optimized automatic speech recognition (ASR) can be used, for example, a model optimized and fine-tuned based on FunASR can be used for real-time conversion. This automatic speech recognition integrates a specific medical terminology database and can convert speech to text after noise reduction preprocessing. In one embodiment, the speech recognition technology can accurately identify the unclear pronunciation of elderly patients.

[0063] For image information, the system employs image recognition technology (e.g., fine-tuning based on RapidOCR) to extract key medical information, such as feature extraction from medical record photos and examination report photos. Image recognition technology can be OCR or a visual recognition model (e.g., a pre-trained GLM-4V model), and both can be used simultaneously or individually. Optionally, structured data from laboratory reports, such as "fasting blood glucose: 7.8 mmol / L↑", can be extracted using OCR technology and associated with the reference range for blood glucose. Simultaneously, a pre-trained visual recognition model (e.g., a pre-trained GLM-4V model) can be invoked to analyze the features of affected area photos. For example, skin lesion images can be analyzed using a visual recognition model to determine "erythema diameter > 5 mm" and "scalp coverage ≥ 30%".

[0064] For user-uploaded attachments, a large text model is used to process them and generate structured text. Optionally, for attachments such as PDF medical records and DICOM images, a pre-trained large language model (LLM, e.g., Qwen2.5-Instruct-32B model) is used for feature extraction, and then the features are structured and mapped to obtain structured clinical features. For example, "5-year history of myocardial infarction" is mapped to the structured field "Past history: myocardial infarction, duration of disease 5 years".

[0065] In one embodiment, a patient's medical profile is formed based on the aforementioned text information, key image information, and structured text. Optionally, the text information, key image information, and structured text results are normalized. Specifically, the speech-to-text, image recognition features, and structured text extracted from attachments are aligned, for example, by associating "chest pain onset time" with the ECG examination timestamp. This ultimately forms a multi-dimensional medical profile encompassing symptoms, signs, auxiliary examinations, and medical history, achieving the integration of multi-source heterogeneous data, improving the accuracy of the patient's medical profile information, and providing a data foundation for subsequent intelligent consultations.

[0066] S200: Conduct a compliance review of the aforementioned annex and obtain the review results;

[0067] The system conducts a comprehensive compliance review of user-uploaded attachments. The review results include at least one of the following: privacy information, formatting specifications, and medical content logic. The review results are then fed back to the user, who can update or modify the attachments based on the feedback.

[0068] In one embodiment, a compliance review is performed on the attachments to obtain the review results, specifically including:

[0069] S201 detects personal privacy information in the text described in the appendix using the detection model and obtains the privacy information review result;

[0070] S202 determines the image clarity and file format standardization according to preset requirements, and obtains the format standardization review result;

[0071] S203 determines the logic between the pre-consultation information and the attachment content according to preset rules, and obtains the medical content logic review result.

[0072] Specifically, in the privacy information review, a privacy detection model (e.g., a pre-trained GLM-4V model) is used to detect whether there is any personal privacy information in the attached text that has not been desensitized, such as ID card number, home address, etc. Optionally, after detecting sensitive information such as ID card number and phone number, regular expression matching is performed on it. For example, the matching rule applicable to ID card is (\d{6}(19|20)\d{9}[0-9X]{1}), and the matching rule applicable to phone number is (1[3-9]\d{9}). When unsensitized data is detected, the system automatically triggers dynamic masking processing, such as replacing "Zhang San" with "Zhang*".

[0073] During the format specification review, the system determines whether the image clarity meets the needs of medical analysis and whether the file format complies with the specifications based on preset requirements. Different rules are used for different file formats. For image attachments, the Sobel operator edge detection is used to calculate the clarity score. For example, the preset requirement is that the image resolution is ≥200dpi to be acceptable. The preset requirements for file formats are that they meet the compliant formats of DICOM and PDF, etc., and are on the preset medical document whitelist; or the file is not encrypted, or is not a compressed file, etc., which are considered risky formats.

[0074] In the logical review of medical content, the system determines the rationality of the logical relationship between the pre-consultation information and the attached content based on preset rules. For example, preset rules may determine whether there are obvious contradictions, temporal contradictions, or reasoning contradictions between the symptom description and the provided examination report. Optionally, the logical review of medical content is based on the construction of a medical knowledge graph, and contradictions are discovered through entity relationship reasoning. For example, if the user's complaint is "no history of diabetes" but the attached laboratory report shows "fasting blood glucose 11.2 mmol / L", a logical conflict is triggered. In this case, the system will perform a temporal consistency check to ensure that the date of the laboratory report is not earlier than the time when the symptoms occurred.

[0075] Finally, at least one of the above audit results will be fed back to the user. For audit logs of automatically de-identified records triggered by privacy leakage risks, specific error codes will be returned for corresponding format violations, such as ERR_RES_200 indicating insufficient resolution. For logical conflicts, a warning mark that requires doctor review will be generated.

[0076] In this embodiment, as Figure 2The triple-layered protection mechanism, as shown in the compliance review, effectively addresses the issue of incomplete information collection during pre-consultation: the privacy review layer employs a dual-track detection model to accurately identify sensitive information, combined with dynamic de-identification technology to meet stringent legal requirements, thus mitigating the risk of medical data leakage at its source; the format review layer sets quantitative standards based on medical diagnostic criteria to ensure that imaging data meets clinical diagnostic needs, significantly improving the usability of medical data; and the medical logic review layer relies on a knowledge graph to build a clinical rule engine, ensuring the accuracy of medical decisions through temporal consistency checks and contradiction detection. These three technical features create a synergistic effect: the privacy protection mechanism ensures the legality of medical data flow, format standardization enhances the reliability of diagnostic support, and clinical logic verification maintains the authenticity of diagnostic evidence. Ultimately, the system can ensure the quality and compliance of collected information, providing a reliable basis for subsequent analysis.

[0077] S300: The step of generating interactive questions based on the review results and the pre-consultation information, and conducting multi-round dialogues based on the interactive questions.

[0078] First, based on the user's selected department or disease type, the system retrieves the corresponding pre-consultation process. Then, based on the attachment review results and the user's answers, along with the provided medical information, the system retrieves the corresponding dialogue process from a pre-configured dialogue template library—specifically, a structured consultation process. Taking cardiology as an example, the system loads a dedicated process template containing core questions such as "nature of chest pain - radiation location - ECG correlation." This template pre-sets key symptom inquiry nodes and attachment triggering rules.

[0079] In one embodiment, generating interactive questions based on the review results and user information specifically includes:

[0080] S301 retrieves the corresponding pre-consultation process based on the user's selection;

[0081] S302 updates the dialogue status based on the audit results and user responses;

[0082] S303 triggers the jump logic based on the updated dialogue state;

[0083] S304 generates a corresponding follow-up question process based on knowledge graphs and user answers.

[0084] The system first retrieves the corresponding dialogue flow based on the user's selection. These templates are categorized by department or disease type and include a set of core questions for different situations. During the dialogue, the system updates the dialogue status based on the attachment review results and the user's answers. The updated dialogue status then triggers redirection logic to achieve a personalized consultation path. The system also generates corresponding follow-up question flows based on a medical knowledge graph and user answers. These follow-up question flows also include requesting the user to upload attachments and answer specific questions at designated nodes.

[0085] When a user selects a specific department or disease, the system automatically invokes the corresponding structured dialogue process. This process includes a set of core questions related to the department or disease and is configured with a dynamic triggering mechanism.

[0086] The system updates the dialogue status based on the attachment review results and user responses. Then, it triggers a jump logic based on the updated dialogue status. Optionally, for example, when the attachment review returns "ST segment elevation on ECG", the myocardial infarction consultation branch is automatically activated, and the basic examination inquiry step is skipped. Secondly, the intelligent decision-making system responds to user responses in real time through jump logic and performs similarity calculation (e.g., cosine similarity). Optionally, when the cosine similarity is ≥0.85, key expressions are identified, such as associating "heartburn" with the entity "angina pectoris". Based on the entity information, a preset path is triggered. If "retrosternal squeezing pain" is selected, the system jumps to the coronary heart disease follow-up question tree.

[0087] In one embodiment, after a user's uploaded medical images (such as CT scans) pass compliance review, the file parsing unit automatically extracts key indicators from the image report (such as "lung nodule diameter = 8mm"). The system will then consider two aspects: First, based on the clinical pathway in the medical knowledge graph that "lung nodules > 8mm require malignancy risk assessment", it will generate targeted follow-up questions such as "Do you have a history of smoking or a family history of lung cancer?"; Second, by comparing the logical contradiction between the user's complaint of "no cough symptoms" and the image conclusion of "ground-glass opacity", it will trigger a clarifying question such as "Have you recently experienced unexplained dry cough?"

[0088] In one embodiment, if the attachment review returns an "image blurry" status, for example, if the image resolution is <200 dpi, the system will first interrupt the current consultation process and drive the digital human to use a magnifying glass gesture animation combined with a voice prompt: "Please re-upload a clearer version of the CT scan, focusing on the 3rd-5th intercostal space." For patients with chronic diseases seeking follow-up visits, the system automatically skips the basic medication inquiry step by associating medication records (such as insulin injection dosage) in historical medical records with the currently uploaded blood glucose monitoring data, and directly generates a dosage adjustment suggestion question: "Have you experienced nocturnal hypoglycemia recently?"

[0089] like Figure 3 As shown, a corresponding follow-up questioning process is generated based on a knowledge graph. The follow-up questioning process includes requesting the upload of attachments and answering questions at a specified node. Optionally, the relationship between related symptoms and diseases is studied based on the knowledge graph relationship, professional questions are dynamically inserted, and attachment requests are initiated at a specified node. For example, in the path "angina pectoris-complications-heart failure", the question "Do you experience paroxysmal nocturnal dyspnea?" is inserted. And when the user mentions "lower limb edema", the instruction "Please upload a recent BNP test report" is triggered.

[0090] In one embodiment, the dialogue process further includes:

[0091] The S3011 attachment upload node is used to request the upload of medical documents related to the current symptoms;

[0092] The S3012 data acquisition node is used to acquire health information of preset categories.

[0093] Specifically, in the dialogue process, the attachment upload node requests related data types, such as test reports and imaging data; the information collection node obtains health information of preset categories and related data types to be collected, such as symptom duration and pain level. When the dialogue reaches the end condition, the system triggers the pre-diagnosis summary generation process.

[0094] Specifically, in the attachment upload node, the system enforces medical data type constraints. For example, the ECG node limits the file type to DICOM or PDF and binds a minimum resolution threshold of 200dpi. The data acquisition node pre-sets structured data acquisition rules, mapping user responses to medical entity slots. For example, the "pain level" field is bound to a VAS score of 0-10, and the "medication history" slot is associated with a generic drug name database. Regular expressions are used to validate the input compliance. For example, blood pressure values ​​must meet the "systolic / diastolic" format.

[0095] Specifically, the system employs a multi-level trigger mechanism to end the consultation. When the core symptom slots are fully filled and key attachments have been approved, the system automatically terminates the consultation process and initiates the summary generation module. For example, for heart-related diseases, once the nature, duration, and radiation location of chest pain are entered, and the ECG report status is "parsed," the system automatically terminates the consultation process and initiates the consultation summary generation module. Optionally, an exception log can be generated for incomplete items; for example, if an allergy history is missing, it can be marked as "requires a doctor's in-person consultation to supplement."

[0096] This dynamic question generation mechanism is achieved through a three-layer collaboration: the dialogue process engine ensures that the consultation framework complies with medical standards, the state decision-maker realizes real-time verification of attachment data and user descriptions, and the knowledge graph-driven module provides evidence-based medicine-supported follow-up questioning chains. Ultimately, this improves the completeness of key indicators collected in pre-consultation, enhances questionnaire efficiency, improves the efficiency of pre-consultation, and increases the accuracy of results.

[0097] S400: Obtain user responses through the multi-turn dialogue, and generate a structured pre-diagnosis summary based on the user responses.

[0098] In one embodiment, the multi-turn dialogue is performed by a digital human virtual assistant, and the process of obtaining user responses through multi-turn dialogue and generating a structured pre-diagnosis summary based on the dialogue results includes:

[0099] S401 determines the digital human virtual assistant based on the pre-consultation information;

[0100] S402 Obtains the user's intent based on the user's response;

[0101] S403 dynamically adjusts the dialogue content based on the user's intent and the user's response.

[0102] In one embodiment, a virtual assistant is obtained based on the pre-consultation information. The system achieves precise digital human adaptation through medical scenario-based customization: First, a multi-dimensional matching algorithm is executed based on the pre-consultation information, combining parameters such as age stratification rules, department attributes, and medical terminal type to obtain a virtual assistant. The optional virtual assistant can be a digital human avatar. The system supports appearance customization; for example, a 3D elderly doctor model is automatically loaded for elderly patients, and a cartoon animal avatar is used for child patients. Then, the color of the white coat is adjusted to match the hospital style.

[0103] The system interacts with users through dialogue guided by digital avatars. First, it obtains a suitable virtual assistant based on pre-consultation information. This process includes matching a corresponding 2D or 3D digital avatar with the user's personal information (such as age, gender, and department of visit). Users can also adjust the digital avatar's appearance according to their preferences. The adjusted digital avatar will then act as a virtual assistant to communicate with the user. During the dialogue, the system analyzes the user's responses to obtain their intent and dynamically adjusts the dialogue content based on this intent and the user's responses.

[0104] In one embodiment, determining a digital human virtual assistant based on the pre-consultation information specifically includes the following steps:

[0105] S4011 matches a corresponding 2D or 3D digital human image based on the user's personal information;

[0106] S4012 users can customize the appearance of their digital avatar to their liking;

[0107] S4013 uses the modified digital human as a virtual assistant.

[0108] Specifically, in the virtual assistant matching process, the system automatically matches a 2D / 3D digital human avatar based on the user's pre-consultation information (including age, department, and other medical characteristics). Medical institutions can pre-configure a specialized avatar library for each department: cardiology is associated with a stethoscope-wielding character, dermatology with a magnifying glass prop for skin lesions, pediatrics with a cartoon animal character and toy props, and orthopedics with a standard stethoscope positioning posture along the right sternal border. The system also loads a medical scene action library, including standard hand gestures for cardiopulmonary auscultation and professional nerve percussion actions, and associates voice synthesis parameters to ensure smooth doctor-patient communication.

[0109] When a patient initiates a consultation, the system intelligently matches an initial image based on pre-consultation information: cartoon characters for children, a silver-haired doctor model for elderly patients, and a professional image with a stethoscope for cardiovascular patients. Users can then customize the image within the framework of medical guidelines: modify facial details to enhance approachability, adjust the white coat color to suit visual preferences, optimize speech speed to improve communication efficiency, and customize gestures to enhance the interactive experience.

[0110] The virtual assistant obtained through the above methods retains medical professionalism by configuring the standard position of the stethoscope in the system, and then incorporates personalized elements by adjusting the digital human's appearance, thereby improving the completion rate of consultations and patients' trust.

[0111] In one embodiment, the dynamic adjustment of the dialogue content specifically includes:

[0112] S4031 obtains the context information of the current dialogue;

[0113] S4032 Obtain the analysis results of the attachment;

[0114] S4033 uses a knowledge graph to reason about the analysis results and the context information to obtain a reasoning result;

[0115] S4034 uses preset dialogue logic to predict the next round of questions based on the reasoning result.

[0116] The process of dynamically adjusting the dialogue content includes: obtaining contextual information of the current dialogue, including the user's previous answers and emotional state; analyzing attachment content and extracting key medical information; using a knowledge graph to comprehensively analyze the attachment analysis results and contextual information to establish the association between symptoms, examination results and possible diseases; and using preset dialogue logic to predict the next most appropriate question based on the analysis results to ensure the relevance and effectiveness of the consultation.

[0117] In one embodiment, based on the corresponding question and answer, the system obtains the user's intent, analyzes the user's answer through a medical intent recognition model, extracts clinical entities by combining a BiLSTM-CRF model, and calculates the emotional anxiety value; for example, "heartache after meals" is analyzed as "postprandial angina" entity, and when the speech spectrum energy is >65dB and the fundamental frequency jitter rate is ≥15%, it is marked as a high anxiety state.

[0118] In one embodiment, the dialogue content is dynamically adjusted based on user intent and responses, and also dynamically adjusted based on medical rules. When a high anxiety state is detected, the digital human activates a "leaning forward + slightly furrowed brows" expression, while inserting reassuring statements and simplifying medical terminology, such as "I understand your anxiety, we will gradually confirm the symptoms." For key medical contradictions, a knowledge graph-driven chain of follow-up questions is automatically triggered. For example, if a user denies a history of diabetes but the attachment shows HbA1c=9.2%, then a chain of follow-up questions is generated: abnormal HbA1c - differential diagnosis - stress hyperglycemia, generating the question "Have you had any major trauma or surgery recently?".

[0119] In one embodiment, during the dynamic dialogue adjustment process, multi-source input data is integrated, including dialogue content, key indicators in attachments, and the system's fill-in-the-blank results for preset questions based on the answers, such as the fasting blood glucose value of 7.8 mmol / L and "chest pain duration = 3 hours" in the test report.

[0120] Then, the above multi-source information is normalized. Specifically, the system calls the medical knowledge graph to process the above information. Specifically, the system links the user's description of "angina pectoris" to the standard term "angina pectoris", and maps "spiculated mass" in the CT image report to "spiculated mass (BI-RADS 4)".

[0121] In one embodiment, for detected logical contradictions, the system inserts a warning marker in the "Items to be Clarified" field and associates it with the original dialogue round number. For example, a user complains of "no cough" but imaging suggests "lung infection." Finally, structured content is filled into a customized template based on the selected department: taking respiratory medicine as an example, based on pre-consultation information, symptom temporal characteristics are summarized, key data from attachments is integrated, and finally, a differential diagnosis list is generated based on knowledge graph reasoning. For example, "cough with sputum appeared 3 days before fever" will be filled into the symptom temporal characteristics; "WBC=15.2×10" in the blood routine test will be filled into the differential diagnosis list. 9 The " / L" mark is used as an outlier and associated with the normal range reference as part of the examination; finally, the differential diagnosis list is generated by inferring "70% probability of community-acquired pneumonia, tuberculosis to be ruled out".

[0122] In one embodiment, when a contradiction arises in the logical verification, the rule engine drives dynamic decision-making, and for non-critical anomalies, it generates tiered follow-up questions based on clinical guidelines. For example: when a user complains of "fatigue for three months" but the test report is from three days ago, a report timeliness warning is triggered; when a critical value (HGB < 9 g / dL) is detected, a red alert question is immediately inserted: "Have you experienced palpitations or fainting? You need to seek emergency medical attention!"; when ferritin < 30 ng / mL, an iron deficiency anemia differential chain is initiated: first ask "Increased menstrual flow?", then check "History of gastrointestinal bleeding?".

[0123] In one embodiment, in a respiratory disease pre-consultation scenario, after a user accesses the system via a mobile terminal, the system first completes identity authentication and loads respiratory medicine-specific resources: including a 2D cartoon digital human character "Xiao U" and a respiratory consultation template containing a core symptom set, with a pre-configured gentle female voice. When the user describes "cold with fever, runny nose, and cough" via voice, the medical ASR engine (a model optimized and fine-tuned based on FunASR) converts it into structured text. The system's pre-trained large model accurately extracts core symptom entities, such as fever, runny nose, and cough. The system then triggers a dynamic follow-up questioning chain, guiding the user to provide key information such as the duration of symptoms, highest body temperature, and nature of the cough step by step. During the interaction, after the user uploads a photo of their tongue, the review process performs triple verification: format compliance is confirmed as a JPG file with no privacy risk; integrity assessment confirms that the image clarity meets diagnostic requirements; and the content parsing module records tongue color characteristics for the doctor's reference. After multiple rounds of dialogue, the system integrates symptom descriptions and attachment information to generate a structured summary. The chief complaint field summarizes core symptoms such as fever, runny nose, and cough; the present illness section organizes the progression of the illness chronologically; and tongue images are stored as attachment IDs. This summary report is written to the hospital's HIS system in real time after user confirmation.

[0124] In another embodiment, regarding the application for pre-consultation for diabetes follow-up visits, targeting chronic disease management scenarios, the system automatically identifies type 2 diabetes patients undergoing follow-up visits using their patient IDs and loads endocrinology-specific configurations, including a 3D realistic doctor image and a diabetes follow-up template. The digital human proactively guides the patient to upload their blood glucose record book. The OCR engine extracts structured data from the image and marks abnormal trends (fasting blood glucose 6.5-7.0 mmol / L showing an upward trend, postprandial blood glucose 9.8-10.5 mmol / L), with a completeness check indicating "missing lunch record." Based on the blood glucose data, the system intelligently skips basic inquiries and directly proceeds to the medication regimen verification and lifestyle assessment stages. When a patient reports blurred vision, the knowledge graph immediately links to the diabetic retinopathy pathway, generating a deep inquiry chain regarding the duration of onset, symptom persistence, and ophthalmological examination history. The system then enters a medical logic verification process, cross-referencing historical EMR data and detecting a correlation between blurred vision and diabetic retinopathy. Finally, a summary is generated, highlighting changes in blood glucose control trends, the emergency indication for blurred vision, the current medication regimen, and recommending a priority consultation with an ophthalmologist within 24 hours. The final report is then pushed to the attending physician via a standardized interface, enabling seamless follow-up consultations.

[0125] In one embodiment, the model in the above embodiment is pre-trained or fine-tuned with corresponding training samples as needed to meet the requirements of the present invention. The method of sample training and the method of fine-tuning can use the methods in the prior art.

[0126] like Figure 4 As shown, this invention integrates multimodal information collection through voice interaction, text analysis, image recognition, and document parsing, combined with a dynamic consultation engine driven by a medical knowledge graph and an intelligent attachment review mechanism, ultimately generating a structured pre-consultation summary. This technology effectively solves the core problems of information fragmentation, rigid processes, and high compliance risks in traditional pre-consultation, significantly improving the completeness and accuracy of medical data collection, while providing doctors with high-value pre-diagnosis decision support.

[0127] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0128] Based on the same inventive concept, this application also provides a pre-consultation system for implementing the pre-consultation method described above. The solution provided by this system is similar to the solution described in the above method; therefore, the specific limitations of the one or more pre-consultation system embodiments provided below can be found in the limitations of the pre-consultation method described above, and will not be repeated here.

[0129] In one embodiment, such as Figure 5 As shown, an AI pre-diagnosis system based on multimodal interaction is provided, including: an acquisition module, an approval module, an interaction module, and a generation module, wherein:

[0130] The acquisition module 500 is used to acquire pre-consultation information provided by the user, wherein the pre-consultation information includes at least one of voice, text, images and attachments;

[0131] The review module 502 is used to conduct a compliance review of the attachments and obtain the review results;

[0132] Interaction module 504 is used to generate interactive questions based on the review results and the pre-diagnosis information, and to conduct multi-round dialogues based on the interactive questions;

[0133] The generation module 506 is used to obtain user answers through the multi-round dialogue and generate a structured pre-diagnosis summary based on the user answers.

[0134] In one embodiment, the acquisition module 500 employs a multi-channel input interface design, capable of simultaneously processing information input in multiple formats. For voice information, the acquisition module captures user speech via a microphone array and performs real-time noise reduction processing; for text information, the system provides an intelligent input box supporting manual input and speech-to-text conversion; for image information, the acquisition module supports uploading images in multiple formats, including common formats such as JPG, PNG, and BMP; for attachment information, the system supports uploading files in formats such as PDF, DOC, and DOCX, with a single attachment size limit of 20MB and a total attachment size not exceeding 100MB. While receiving information, the acquisition module performs preliminary classification and labeling of the information to prepare for subsequent processing.

[0135] In one embodiment, the review module 502 first checks whether the attachment format conforms to the types supported by the system, and then performs content security review, including three stages: sensitive information detection, prohibited content identification, and virus scanning. Sensitive information detection mainly targets and protects personal privacy data, such as ID card numbers and bank card numbers; prohibited content identification uses deep learning algorithms to analyze the attachment content to identify whether it contains illegal or non-compliant content; virus scanning ensures that the attachment does not contain malicious code. The review module generates a detailed review report, including approved items, warning items, and rejected items, and provides a comprehensive review result. In a preferred embodiment, the review module also performs a medical relevance assessment of the attachment content to determine whether the uploaded content is related to medical consultation, improving the system's professional targeting.

[0136] In one embodiment, the interaction module 504 employs an adaptive question generation strategy, dynamically constructing a consultation path based on initial information provided by the user and feedback from the review module. The interaction module incorporates a medical knowledge graph and symptom decision tree, enabling it to generate targeted follow-up questions based on the user's described symptoms and uploaded materials. Question generation follows a principle of moving from general to specific, and from primary symptoms to secondary symptoms, ensuring clear consultation logic. The interaction module supports multiple interaction methods, including text dialogue, voice interaction, and a graphical selection interface, allowing users to choose according to their preferences. During question generation, the system considers the user's age, gender, medical history, and other personal characteristics, providing a personalized question sequence. In a preferred embodiment, the interaction module also features emotion recognition, capable of judging the user's emotional state based on tone, word choice, and expression, adjusting the expression and depth of questions accordingly to provide a more humanized interactive experience.

[0137] In one embodiment, the generation module 506 employs a progressive information collection strategy, supplementing and refining the user's health information in each round of dialogue. During multiple rounds of dialogue, the system dynamically adjusts the depth and breadth of questions to ensure sufficient diagnostic information is collected. After the dialogue ends, the generation module integrates and analyzes all collected information to generate a structured pre-diagnosis summary in standard medical record format, including chief complaint, present illness, past medical history, family history, and personal history. The pre-diagnosis summary uses a hierarchical display method, including three levels: key information summary, detailed symptom description, and possible disease directions, facilitating a doctor's quick understanding of the patient's condition. The generation module also performs a reliability assessment of the collected information, marking any insufficient or unclear information to highlight key points for the doctor to confirm during the formal consultation. In a preferred embodiment, the generation module also provides a health advice function, offering preliminary health guidance and medical advice to the user based on the pre-diagnosis results, but explicitly stating that these suggestions do not constitute a medical diagnosis and require further confirmation by a professional physician.

[0138] This pre-consultation system achieves intelligent and standardized pre-consultation processes through the collaborative work of four core modules: multimodal information acquisition, intelligent compliance review, adaptive interactive question and answer, and structured summary generation. This effectively improves the efficiency of medical resource utilization and enhances the patient experience. While protecting user privacy, the system provides doctors with structured and comprehensive patient information, laying the foundation for subsequent formal diagnosis and treatment.

[0139] Each module in the aforementioned pre-diagnosis system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0140] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the content of any embodiment of the pre-diagnosis method described above.

[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the content of any embodiment of the pre-diagnosis method described above.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the content of any embodiment of the pre-diagnosis method described above.

[0143] This instruction manual also provides Figure 6 The diagram shows a schematic structural representation of the electronic device. Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The pre-diagnosis method described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0144] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0145] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0146] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0147] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0148] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0150] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0151] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0152] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0153] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0154] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0156] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0158] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0159] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A pre-diagnosis method, characterized by, The method comprises the following steps: obtaining pre-diagnosis information provided by a user, the pre-diagnosis information comprising at least one of voice, text, pictures and attachments; performing compliance audit on the attachments to obtain an audit result; generating interactive questions based on the audit result and the pre-diagnosis information, and performing multi-round dialogue based on the interactive questions; obtaining user answers through the multi-round dialogue, and generating a structured pre-diagnosis summary according to the user answers.

2. The method of claim 1, wherein, The obtaining of the pre-diagnosis information provided by the user specifically comprises: for the voice, obtaining text information by using voice recognition technology; for the pictures, obtaining key information of the pictures by using picture recognition technology; for the attachments, generating structured text by using a text large model.

3. The method of claim 1, wherein, The audit result comprises at least one of privacy information, format specification and medical content logic, and the performing of the compliance audit on the attachments to obtain the audit result comprises: detecting personal privacy information in the text of the attachments according to a detection model to obtain a privacy information audit result; judging the picture definition and the file format specification according to preset requirements to obtain a format specification audit result; judging the logic of the pre-diagnosis information and the attachment content according to preset rules to obtain a medical content logic audit result.

4. The method of claim 1, wherein, The generating of the interactive questions based on the audit result and the pre-diagnosis information, and the performing of the multi-round dialogue based on the interactive questions comprise: obtaining a pre-diagnosis process corresponding to the user selection; updating the dialogue state based on the audit result and the user answers; triggering a jump logic according to the updated dialogue state; generating a corresponding follow-up process based on a knowledge graph and the user answers.

5. The method of claim 4, wherein, The pre-diagnosis process further comprises: an attachment uploading node for requesting uploading of a medical file associated with a current symptom; a data collection node for obtaining health information of a preset category.

6. The method of claim 1, wherein, The multi-round dialogue is performed by a digital human virtual assistant, and the obtaining of the user answers through the multi-round dialogue and the generating of the structured pre-diagnosis summary based on the dialogue result comprise: determining the digital human virtual assistant according to the pre-diagnosis information; obtaining a user intention according to the user answers; dynamically adjusting dialogue content based on the user intention and the user answers.

7. The method of claim 6, wherein, The dynamically adjusting of the dialogue content based on the user intention and the user answers comprises: obtaining context information of the current dialogue; obtaining an analysis result of the attachments; using a knowledge graph to reason the analysis result and the context information to obtain a reasoning result; predicting a next round question using a preset dialogue logic according to the reasoning result.

8. A pre-diagnosis system, characterized by, The method comprises the following steps: an obtaining module for obtaining pre-diagnosis information provided by a user, the pre-diagnosis information comprising at least one of voice, text, pictures and attachments; an auditing module for performing compliance audit on the attachments to obtain an audit result; an interactive module for generating interactive questions based on the audit result and the pre-diagnosis information, and performing multi-round dialogue based on the interactive questions; a generating module for obtaining user answers through the multi-round dialogue, and generating a structured pre-diagnosis summary according to the user answers. 9.An electronic device comprising a memory and a processor, wherein the memory stores a computer program. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images