Intelligent pre-inquiry method and device, intelligent equipment and storage medium

By obtaining the basic information of patients and generating target digital people for personalized interaction, using large language models for semantic understanding and reasoning, combined with hospital government information, the problem of insufficient personalization of the existing pre-diagnosis model is solved, and the pre-diagnosis model is improved Accuracy and professionalism enhance the patient experience.

CN118942736APending Publication Date: 2024-11-12GUANGDONG LAB OF ARTIFICIAL INTELLIGENCE & DIGITAL ECONOMY (SZ)
View PDF 0 Cites 8 Cited by

Patent Information

Application Number
CN202411030663.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing pre-diagnosis model lacks personalization, resulting in low efficiency and effectiveness, making it difficult to meet the personalized needs of patients.

Method used

By obtaining the basic information of the patient, generating target numbers for personalized interactions with the patient, using a large language model to semantic understanding and reasoning of the disease description, and generating pre-diagnosis results based on the hospital's government information.

Benefits of technology

It improves the accuracy and professionalism of pre-diagnosis, enhances the patient experience, and improves the diagnosis and treatment efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942736A_ABST
    Figure CN118942736A_ABST
Patent Text Reader

Abstract

The invention is suitable for the technical field of artificial intelligence, and provides an intelligent pre-inquiry method and device, intelligent equipment and a storage medium, and the method comprises the steps: obtaining the basic information of a patient; according to the basic information, a target digital person is generated, and the target digital person is used for interacting with the patient according to an interaction strategy and guiding the patient to provide illness state description; performing semantic understanding and reasoning on the basic information and the condition description by using a first large language model to generate an initial pre-inquiry suggestion; and combining the basic information and the initial pre-inquiry suggestion with hospital government affair information by using a second big language model to generate a pre-inquiry result of the patient. According to the method and the device, personalized and professional pre-inquiry service can be provided for the patient, the pre-inquiry efficiency and the pre-inquiry effectiveness are improved, and the patient experience is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an intelligent pre-diagnosis method, apparatus, intelligent device and storage medium. Background Art

[0002] With the increase in population, medical resources are becoming increasingly scarce. In the traditional medical consultation plan, doctors directly ask patients about their symptoms when they are sick or unwell to diagnose their condition, or they consult patients through remote video consultation. Whether it is a face-to-face consultation or a remote video consultation, it takes up a lot of doctors' energy, the consultation process is long, and it is particularly easy to miss something. Therefore, the pre-consultation model was born. In the modern medical consultation plan, pre-consultation before diagnosis can help doctors quickly understand the patient's condition in advance, and make relevant preparations and arrangements in advance, improve the doctor's diagnosis and treatment efficiency, and improve work efficiency.

[0003] Due to the diversity of patients' personalities, the effectiveness of a unified and standardized pre-consultation method is not high. In view of this, how to provide patients with personalized and professional pre-consultation services and improve the efficiency and effectiveness of pre-consultation is a problem that needs to be considered at present. Summary of the invention

[0004] The embodiments of the present application provide an intelligent pre-consultation method, apparatus, intelligent device and storage medium, which can provide patients with personalized and professional pre-consultation services, improve the efficiency and effectiveness of pre-consultation, and enhance the patient experience.

[0005] In a first aspect, an embodiment of the present application provides an intelligent pre-diagnosis method, the intelligent pre-diagnosis method comprising:

[0006] Obtain basic information of patients;

[0007] Generate a target digital person based on the basic information, wherein the target digital person is used to interact with the patient according to an interaction strategy to guide the patient to provide a description of the condition;

[0008] Using the first language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate initial pre-diagnosis suggestions;

[0009] The basic information and the initial pre-consultation suggestion are combined with hospital government affairs information using a second language model to generate a pre-consultation result for the patient.

[0010] In a possible implementation manner of the first aspect, the basic information includes age and gender, and the step of obtaining the basic information of the patient includes:

[0011] obtaining a facial image of the patient;

[0012] The facial image is input into a pre-trained multi-task convolutional neural network model, and the multi-task convolutional neural network model is used to identify the age and gender of the patient.

[0013] In a possible implementation of the first aspect, the step of using the first large language model to perform semantic understanding and reasoning on the basic information and the condition description to generate an initial pre-diagnosis suggestion includes:

[0014] Fill the basic information and the condition description into a preset prompt template to obtain a prompt;

[0015] The prompt is input into the first large language model, guiding the large language model to perform semantic understanding and reasoning on the basic information and the condition description, and generating an initial pre-diagnosis suggestion.

[0016] In a possible implementation of the first aspect, the step of generating a target digital human according to the basic information includes:

[0017] According to the basic information of the patient, selecting an initial digital human model corresponding to the basic information from a predefined digital human model library;

[0018] determining an interaction style corresponding to the patient;

[0019] A target digital human is generated based on the interaction style and the initial digital human model.

[0020] In a possible implementation manner of the first aspect, the method further includes:

[0021] Identifying the emotional state of the patient during the interaction with the target digital human;

[0022] According to the emotional state, the interaction strategy of the target digital person and the patient is adjusted.

[0023] In a possible implementation of the first aspect, the step of identifying the emotional state of the patient during the interaction with the target digital human includes:

[0024] Acquire the facial image of the patient and the voice information of the patient during the interaction;

[0025] Inputting the facial image of the patient during the interaction into a facial expression recognition model to recognize the facial emotion of the patient;

[0026] Inputting the patient's voice information during the interaction into a voice emotion recognition model to identify the patient's voice emotion state;

[0027] The facial expression is combined with the voice emotion state to determine the patient's emotional state.

[0028] In a possible implementation of the first aspect, the step of adjusting the interaction strategy between the target digital human and the patient according to the emotional state includes:

[0029] According to the emotional state, at least one of the interaction style, interaction guide sentence, interaction rhythm and interaction animation of the target digital human is adjusted.

[0030] In a second aspect, an embodiment of the present application provides an intelligent pre-diagnosis device, the intelligent pre-diagnosis device comprising:

[0031] An information acquisition unit, used to obtain basic information of the patient;

[0032] An intelligent interaction unit, used to generate a target digital person according to the basic information, wherein the target digital person is used to interact with the patient according to an interaction strategy and guide the patient to provide a description of the condition;

[0033] An initial suggestion generating unit, configured to use a first language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate an initial pre-diagnosis suggestion;

[0034] The pre-consultation result generating unit is used to combine the basic information and the initial pre-consultation suggestion with the hospital government affairs information by using the second language model to generate the pre-consultation result of the patient.

[0035] In a third aspect, an embodiment of the present application provides an intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the intelligent pre-diagnosis method as described in the first aspect above is implemented.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intelligent pre-diagnosis method as described in the first aspect above is implemented.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a smart device, the smart device executes the smart pre-diagnosis method as described in the first aspect above.

[0038] In an embodiment of the present application, the smart device obtains the basic information of the patient, generates a target digital person based on the basic information, uses the target digital person to conduct personalized interaction with the patient, and guides the patient to provide a description of the condition, which is conducive to helping the patient relax and improve the accuracy and effectiveness of the condition description. Then, the first language model is used to perform semantic understanding and reasoning on the basic information and the condition description, and professional initial pre-consultation suggestions are quickly generated, which can improve the accuracy of the pre-consultation. The second language model is then used to combine the basic information and the initial pre-consultation suggestions with the hospital's government affairs information to generate more professional and comprehensive pre-consultation results for the patient, improve the professionalism and effectiveness of the pre-consultation, and enhance the patient experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 is a schematic diagram of a pre-diagnosis system using the intelligent pre-diagnosis method provided in an embodiment of the present application;

[0041] Figure 2 is a flowchart of the implementation of the intelligent pre-diagnosis method provided in the embodiment of the present application;

[0042] Figure 3 This is a specific implementation flow chart of step S201 of the intelligent pre-diagnosis method provided in an embodiment of the present application;

[0043] Figure 4 This is a specific implementation flow chart of generating a target digital person in the intelligent pre-diagnosis method provided in the embodiment of the present application;

[0044] Figure 5 This is a specific implementation flow chart of identifying the patient's condition in the intelligent pre-diagnosis method provided in the embodiment of the present application;

[0045] Figure 6 is a structural block diagram of the intelligent pre-diagnosis device provided in an embodiment of the present application;

[0046] Figure 7 It is a schematic diagram of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0049] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0051] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0052] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0053] It should be understood that the intelligent pre-diagnosis method provided in the embodiment of the present application is applicable to various types of intelligent devices and servers that need to perform intelligent pre-diagnosis, and the intelligent devices may specifically include mobile phones, tablet computers, wearable devices, laptop computers, and desktop computers, etc. The embodiment of the present application does not impose any restrictions on the specific types of intelligent devices and servers.

[0054] The following specifically describes an application scenario of an intelligent pre-diagnosis method provided by an embodiment of the present application, and the specific description is as follows:

[0055] An intelligent pre-diagnosis method provided in an embodiment of the present application is applied to a pre-diagnosis system.

[0056] Figure 1 A schematic diagram of a pre-diagnosis system provided in an embodiment of the present application is shown. Figure 1 As shown, the pre-diagnosis system at least includes an intelligent device 10 and a remote server 11, wherein the intelligent device 10 includes a display device 101 and a collection device 102, and the intelligent device 10 also has a built-in processing control module 103. Among them, the above-mentioned display device 101 is used to display the target digital person, the above-mentioned collection device 102 is used to collect patient image information and voice information, and the above-mentioned processing control module 103 can be connected to the remote server 11 for communication, and is used to execute the intelligent pre-diagnosis method provided in the embodiment of the present application. The number of the above-mentioned collection devices 102 can be one or more. If the number of the collection device 102 is one, the collection device 102 can also be set on the center line of the plane where the display device 101 of the intelligent device 10 is located; if the number of the collection device 102 is two or more, the multiple collection devices 102 can be symmetrically distributed on the above-mentioned center line, and the specific placement of the collection device 102 can be determined according to the actual situation, which is not limited here. The above-mentioned remote server 11 can be a hospital server.

[0057] In some implementations, the acquisition device 102 may be a built-in device of the smart device 10 or an independent external device, such as a smart camera, which is connected to the smart device 10 via an external interface and sends the acquired information to the processing control module 103 in the smart device 10.

[0058] Figure 2 The implementation process of the intelligent pre-diagnosis method provided in the embodiment of the present application is shown, and the method flow includes steps S201 to S204. The specific implementation principle of each step is as follows:

[0059] Step S201: Obtain basic information of the patient.

[0060] The basic information is the basic personal information of the patient, such as age, gender, etc. In this embodiment, the basic information can be used to determine the personalized target digital person corresponding to the patient.

[0061] In a possible implementation, the smart device can obtain basic information actively provided by the patient in the form of voice or text. The basic information is provided by the patient himself, which can ensure the accuracy and reliability of the basic information.

[0062] As a possible implementation of this application, Figure 3 The specific implementation process of step S201 of the intelligent pre-diagnosis method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0063] A1: Obtain the facial image of the patient. The facial image of the patient can be captured by using the camera of the smart device.

[0064] A2: Input the facial image into a pre-trained multi-task convolutional neural network model, and use the multi-task convolutional neural network model to identify the age and gender of the patient.

[0065] The smart device can collect facial images of patients through the camera. After obtaining the facial images, it uses the pre-trained multi-task convolutional neural network model to simultaneously perform gender recognition and age prediction. The multi-task convolutional neural network model includes a first task sub-model for gender recognition and a second task sub-model for age prediction. Among them, gender recognition is a binary classification problem, and the first task sub-model can be trained using the cross entropy loss function; the second task sub-model can be combined with the deep learning (Deep Label Distribution Learning, DLDL) method for age prediction to improve the prediction accuracy.

[0066] In this embodiment, the smart device obtains the patient's facial image and uses a pre-trained multi-task convolutional neural network model to recognize and process the facial image to determine the patient's age and gender without the patient actively providing it. This can avoid situations where the patient is unable or inconvenient to actively provide the information. In addition, the patient's age and gender can be automatically identified, making the pre-consultation service more humane and intelligent.

[0067] Step S202: Generate a target digital human based on the basic information. The target digital human is used to interact with the patient according to an interaction strategy to guide the patient to provide a description of the condition.

[0068] The target digital person is a personalized digital person corresponding to the basic information of the patient, and the interaction strategy is a personalized interaction strategy for the patient. In some implementations, there is a corresponding relationship between the interaction strategy and the digital person, and the corresponding interaction strategy can be determined based on the determined digital person. This embodiment uses the target digital person corresponding to the patient to interact with the patient based on the interaction strategy, which can effectively guide the patient to provide a description of the condition.

[0069] In this embodiment, during the interaction between the target digital person and the patient, the patient can be guided to provide a description of the condition in the form of text, pictures, voice, or video.

[0070] As a possible implementation of this application, Figure 4 A specific implementation process of generating a target digital person according to the basic information in the intelligent pre-diagnosis method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0071] B1: According to the basic information of the patient, an initial digital human model corresponding to the basic information is selected from the predefined digital human model library. The predefined digital human model library includes a variety of predefined initial digital human models, and different initial digital human models have different digital human images. Digital human images include cartoons, animations, real people, etc. For example, for pediatric patients, a cartoon digital human can be used to increase affinity; for adult patients, a real-life simulated digital human can be used to enhance professionalism.

[0072] B2: Determine the interaction style corresponding to the patient. The interaction style includes but is not limited to interaction speed, interaction wording and timbre. In a possible implementation, the interaction style also includes dialect type.

[0073] In some implementations, the interaction style corresponding to the patient can be determined directly based on the patient's age. The patient's age range is different, and the corresponding interaction style may be different. For example, for patients in the children's age range, a first interaction style with a slow interaction speed, soft wording, and cartoon timbre can be determined; for patients in the youth age range, a second interaction style with a fast interaction speed, serious wording, and male timbre can be determined; for patients in the elderly age range, a third interaction style with a slow interaction speed, soft wording, and female timbre can be determined.

[0074] In this embodiment, after detecting the patient's first voice message, the smart device performs dialect type recognition and analysis on the first voice message to determine its dialect type, and uses the dialect type as the dialect type for the target digital person to interact with the patient.

[0075] In other embodiments, the display screen of the smart device may provide a variety of interaction style options, and the patient may select the interaction style by himself / herself, and the interaction style of the target digital person may be determined based on the patient's selection.

[0076] B3: Generate a target digital human based on the interaction style and the initial digital human model.

[0077] In this embodiment, the initial digital human model is determined based on the basic information of the patient, that is, the image of the target digital human is determined, and then the target digital human's interactive speech speed, interactive wording, timbre and even dialect type in the process of interacting with the patient are determined based on the interactive style selected by the patient. The image of the target digital human and the interactive style in the interactive process are different for different genders and ages of patients. Using the patient's exclusive personalized digital human to guide the patient's interaction and guide the patient to provide a description of the condition is conducive to improving the efficiency of interaction and the effectiveness of the description of the condition.

[0078] In a possible implementation, the smart device further identifies the emotional state of the patient during the interaction with the target digital human, and adjusts the interaction strategy of the target digital human with the patient according to the emotional state. The interaction strategy includes interaction guide sentences, interaction rhythm, and interaction animation.

[0079] The smart device adjusts at least one of the interaction style, interaction guide sentence, interaction rhythm and interaction animation of the target digital person according to the emotional state.

[0080] Exemplarily, when the patient has negative emotions such as anxiety and tension, the target digital person will adjust the interaction strategy in time, and adjust at least one of the interaction style, interaction guide statements, interaction rhythm and interaction animation during the interaction with the patient. For example, the target digital person will switch to a gentler and more empathetic interaction style to create a safe and comfortable consultation atmosphere. The target digital person will slow down the interaction rhythm, give the patient enough time and space to express their feelings and needs, and avoid emotional factors interfering with the quality of information collection. The target digital person can also provide interactive guidance statements containing targeted emotional management suggestions to guide patients to regulate their emotions, such as deep breathing, relaxation training, etc., to help patients restore a calm state. The target digital person can also help patients relieve their emotions by playing interactive animations corresponding to the patient's emotional state. Through the adjustment of the interaction strategy, the target digital person can not only give patients emotional support and encouragement, but also ensure the smooth progress of the pre-consultation process.

[0081] On the contrary, when the patient's emotional state is relatively stable and positive, the target digital human can adjust the interaction style and adopt an interaction style that matches the patient's current emotional state to interact with the patient. The target digital human uses more relaxed and friendly language to create a good pre-consultation atmosphere and enhance trust and communication with the patient. In some implementations, while ensuring the patient's comfort, the target digital human can also appropriately speed up the consultation rhythm, improve the consultation efficiency, make full use of the patient's active cooperation, and efficiently and accurately collect key condition descriptions.

[0082] As a possible implementation of this application, Figure 5A specific implementation process of identifying the emotional state of the patient during the interaction between the patient and the target digital human in the intelligent pre-diagnosis method provided in the embodiment of the present application is shown, and is described in detail as follows:

[0083] C1: Acquire the facial image and voice information of the patient during the interaction process.

[0084] C2: Inputting the facial image of the patient during the interaction into a facial expression recognition model to recognize the facial emotion of the patient. Facial emotion can be determined based on emotion category and emotion intensity.

[0085] C3: Inputting the patient's voice information during the interaction into a voice emotion recognition model to identify the patient's voice emotion state.

[0086] C4: Fusion the facial expression with the voice emotion state to determine the patient's emotional state.

[0087] In this embodiment, the emotional state is determined based on the facial emotion and the voice emotion state. The patient's facial emotion is identified using a facial expression recognition model, the patient's voice emotion state is identified using a voice emotion recognition model, and then the facial emotion and the voice emotion state are integrated to accurately determine the patient's emotional state.

[0088] Among them, the facial expression recognition model can be used, such as FER, EmotionNet, etc., which can identify the patient's emotional category (such as happiness, surprise, sadness, disgust, etc.) and emotional intensity by analyzing facial muscle movements; the speech emotion recognition model can be used, such as EmoAudioNet, Speech Emotion Recognition, etc., which can judge the patient's emotional state by analyzing the rhythm, sound intensity, timbre and other characteristics of the speech. The results of facial expression and speech analysis are then integrated to obtain the patient's emotional state.

[0089] In a possible implementation, the patient's facial image and voice information are input into a multimodal emotion recognition model, and the multimodal emotion recognition model is used to identify the patient's emotional state.

[0090] In this embodiment, the above multimodal emotion recognition model is constructed and trained. The specific implementation process includes:

[0091] 1. Data preparation: Collect and annotate emotion datasets containing multimodal data such as facial expressions, speech, and text, ensuring that the datasets cover different emotion categories and have sufficient data volume and diversity.

[0092] 2. Modal feature extraction: For each modal data, design an independent feature extractor, where:

[0093] Facial expression: Use a convolutional neural network (CNN) based on the attention mechanism to extract facial expression features. Based on the traditional CNN, the attention mechanism is introduced to allow the model to adaptively focus on the key areas of the face and extract more discriminative features;

[0094] Speech information: Deep convolutional neural network (DCNN) is used to extract the acoustic and prosodic features of speech. Residual connections and pooling layers are added to DCNN to enhance the model's ability to extract speech emotion features.

[0095] Text information: Use pre-trained language models such as BERT to extract the semantic and sentiment features of the text. In the fine-tuning stage, introduce sentiment feature auxiliary loss to enable the model to learn more accurate sentiment representation.

[0096] 3. Modality Adaptive Alignment: To alleviate the impact of differences in the distribution of different modal data, a modality adaptive alignment module is designed. A modality alignment method based on adversarial learning is introduced. By training the modality discriminator and feature extractor, different modalities are aligned in a shared space to reduce the distribution gap between modalities. A modality alignment strategy based on multi-kernel learning is designed to learn a kernel function, map different modal data to a high-dimensional space, and minimize their distance in the kernel space to achieve modality adaptive alignment.

[0097] 4. Cross-modal fusion: Design innovative cross-modal fusion mechanisms to make full use of the complementary information of each modality. In this embodiment, a dynamic fusion strategy based on the attention mechanism is proposed. The attention network dynamically assigns weights to each modality, so that the model can adaptively adjust the contribution of each modality according to the emotional state of the input. In this embodiment, a tensor fusion method is also introduced to construct the features of different modalities into high-order tensors. Through tensor decomposition and low-rank constraints, high-order correlations between modalities are mined to obtain a comprehensive and compact emotional representation.

[0098] 5. Modality Drop Mechanism: To further improve the robustness and generalization ability of the model, the Modality Drop mechanism is introduced. During the training process, the features of certain modalities are randomly discarded, forcing the model to learn to use the information of other modalities for sentiment reasoning. This mechanism can alleviate overfitting and improve the performance of the model in the case of missing modalities.

[0099] In this embodiment, a multimodal emotion recognition model is constructed, and the multimodal domain adaptation method and cross-modal fusion are combined. In the modal separation representation learning stage, the model analyzes the relationship between modal invariance and specific modal subspace features in different modal data, and explores the correlation between modalities. At the same time, the multimodal emotion recognition model also needs to solve the distribution gap and information redundancy problems between different modalities. Through domain adaptation methods, such as adversarial learning and multi-core learning, the impact of the distribution differences of different modal data can be alleviated and the generalization ability of the model can be improved. In the cross-modal fusion stage, effective fusion mechanisms are designed, such as attention mechanisms, tensor fusion, etc., which can make full use of the emotional features extracted from each modality, eliminate information redundancy to the maximum extent, and form a comprehensive and accurate emotional representation.

[0100] Step S203: using the first language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate initial pre-diagnosis suggestions.

[0101] The first language model is used to generate initial pre-consultation suggestions for patients. It is a large language model (LLM) used in the medical field. LLM is a language model based on deep learning technology that can process large amounts of text data and learn the grammar, semantics, and contextual information of the language. The first language model can be a medical GPT.

[0102] As a possible implementation of the present application, the basic information and the condition description are filled into a preset prompt template to obtain a prompt, and the prompt is input into the first large language model to guide the large language model to perform semantic understanding and reasoning on the basic information and the condition description to generate initial pre-diagnosis suggestions.

[0103] The first language model mentioned above can also answer medical and health-related questions for patients based on the medical knowledge base.

[0104] Exemplarily, the text corpus of the entire knowledge base is divided into multiple blocks; each block is converted into a vector using an embedding model; all vectors are stored in a vector database; the questions / queries that the patient wants to ask are embedded using the same embedding model used by the embedding knowledge base itself and converted into vectors; a query is run in the index of the vector database using the generated vectors to return the top k most similar vectors in a given embedding / latent space; the question / query and the retrieved text block are combined into prompts according to a template; the prompts are passed to a large language model (LLM), instructing the LLM to answer the given question using only the provided context. The LLM generates an answer based on the prompt and returns it to the patient.

[0105] In this embodiment, by pre-training on large-scale medical literature, electronic medical records and other multi-source heterogeneous data, the language expression and knowledge representation in the medical field are learned. On this basis, fine-tuning is performed for specific application tasks such as disease diagnosis and medical question and answer, so that the large language model can infer the possible diseases of the patient based on the condition description information provided by the patient through knowledge reasoning and semantic understanding, and give disease classification, symptom identification, etc., to generate initial pre-consultation suggestions, which can not only improve the efficiency of pre-consultation, but also improve the effectiveness of pre-consultation.

[0106] Step S204: using the second largest language model to combine the basic information and the initial pre-diagnosis suggestion with the hospital government affairs information to generate a pre-diagnosis result for the patient.

[0107] The second language model is used to generate the patient's pre-consultation results. The pre-consultation results include medical advice and pre-consultation reports. The difference between the second language model and the first language model mentioned above is that the second language model is a large language model that focuses on the learning and application of knowledge in hospital management and medical procedures. In this embodiment, the differentiated model can be customized using private data security, and the internal knowledge base of each hospital clinic and doctor can be constructed using RAG technology. Through pre-training of hospital administrative documents, medical guidelines and other materials, the second language model can master the relevant knowledge of hospital operations. In the fine-tuning process of the medical advice task, the second language model takes the patient's basic information, preliminary pre-consultation results, etc. as input, combined with the hospital's department settings, doctor's expertise and other factors, to generate the patient's pre-consultation results. The pre-consultation results include personalized medical advice, such as recommended medical departments, specialists, examination / treatment plans, etc., so as to intelligently guide the patient's subsequent medical behavior. The pre-consultation results also include pre-consultation reports, which can be generated based on medical advice, so that the patient's attending physician can understand the patient's actual situation in advance before treating the patient. The second largest language model can be hospital government affairs GPT.

[0108] In this embodiment, after comprehensively analyzing the patient's basic information and the description of the patient's condition in various forms such as text, pictures, voice, and video, an initial pre-diagnosis suggestion is given, and then the patient's basic information and the initial pre-diagnosis suggestion are combined to generate a medical recommendation, and a pre-diagnosis report is generated for the patient's attending physician. The pre-diagnosis report integrates the analysis results of the medical GPT and the hospital government affairs GPT, automatically generates a structured and visual pre-diagnosis report, comprehensively presents the patient's basic information, chief complaint, relevant medical history, preliminary pre-diagnosis results, and provides a multi-dimensional correlation analysis of the disease, assisting doctors to quickly understand the patient's condition, improve diagnostic ideas, and develop personalized diagnosis and treatment plans, thereby improving diagnostic efficiency and accuracy.

[0109] In an embodiment of the present application, the smart device obtains the basic information of the patient, generates a target digital person based on the basic information, uses the target digital person to conduct personalized interaction with the patient, and guides the patient to provide a description of the condition, which is conducive to helping the patient relax and improve the accuracy and effectiveness of the condition description. Then, the first language model is used to perform semantic understanding and reasoning on the basic information and the condition description, and professional initial pre-consultation suggestions are quickly generated, which can improve the accuracy of the pre-consultation. The second language model is then used to combine the basic information and the initial pre-consultation suggestions with the hospital's government affairs information to generate more professional and comprehensive pre-consultation results for the patient, improve the professionalism and effectiveness of the pre-consultation, and enhance the patient experience.

[0110] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0111] Corresponding to the intelligent pre-diagnosis method described in the above embodiment, Figure 6 A structural block diagram of an intelligent pre-diagnosis device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0112] Reference Figure 6 The intelligent pre-diagnosis device comprises: an information acquisition unit 61, an intelligent interaction unit 62, an initial suggestion generation unit 63, and a pre-diagnosis result generation unit 64, wherein:

[0113] An information acquisition unit 61 is used to acquire basic information of the patient;

[0114] An intelligent interaction unit 62 is used to generate a target digital person according to the basic information, and the target digital person is used to interact with the patient according to an interaction strategy to guide the patient to provide a description of the condition;

[0115] An initial suggestion generating unit 63 is used to use the first language model to perform semantic understanding and reasoning on the basic information and the condition description to generate an initial pre-diagnosis suggestion;

[0116] The pre-diagnosis result generating unit 64 is used to combine the basic information and the initial pre-diagnosis suggestion with the hospital government affairs information by using the second largest language model to generate the pre-diagnosis result of the patient.

[0117] As a possible implementation of the present application, the information acquisition unit 61 includes:

[0118] An image acquisition module, used for acquiring a facial image of a patient;

[0119] The age and gender recognition module is used to input the facial image into a pre-trained multi-task convolutional neural network model, and use the multi-task convolutional neural network model to identify the age and gender of the patient.

[0120] As a possible implementation of the present application, the initial suggestion generating unit 63 includes:

[0121] A prompt generation module, used to fill the basic information and the condition description into a preset prompt template to obtain a prompt;

[0122] The initial suggestion generation module is used to input the prompt into the first large language model, guide the large language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate initial pre-diagnosis suggestions.

[0123] As a possible implementation of the present application, the intelligent interaction unit 62 includes:

[0124] An initial model selection module is used to select an initial digital human model corresponding to the basic information from a predefined digital human model library according to the basic information of the patient;

[0125] A style acquisition module, used to determine the interaction style corresponding to the patient;

[0126] A digital human generation module is used to generate a target digital human based on the interaction style and the initial digital human model.

[0127] As a possible implementation of the present application, the above-mentioned intelligent pre-diagnosis device further includes:

[0128] An emotional state recognition unit, used to recognize the emotional state of the patient during the interaction with the target digital human;

[0129] An interaction strategy adjustment unit is used to adjust the interaction strategy of the target digital person and the patient according to the emotional state.

[0130] As a possible implementation of the present application, the above-mentioned emotional state recognition unit includes:

[0131] An interaction information acquisition module, used to acquire the facial image of the patient and the voice information of the patient during the interaction;

[0132] An expression recognition module, used to input the facial image of the patient during the interaction into a facial expression recognition model to recognize the facial emotions of the patient;

[0133] An emotion recognition module, used to input the patient's voice information during the interaction into a speech emotion recognition model to recognize the patient's speech emotion state;

[0134] The emotional state determination module is used to fuse the facial expression with the voice emotional state to determine the emotional state of the patient.

[0135] As a possible implementation manner of the present application, the above-mentioned interaction strategy adjustment unit is specifically used for:

[0136] According to the emotional state, at least one of the interaction style, interaction guide sentence, interaction rhythm and interaction animation of the target digital human is adjusted.

[0137] In an embodiment of the present application, the smart device obtains the basic information of the patient, generates a target digital person based on the basic information, uses the target digital person to conduct personalized interaction with the patient, and guides the patient to provide a description of the condition, which is conducive to helping the patient relax and improve the accuracy and effectiveness of the condition description. Then, the first language model is used to perform semantic understanding and reasoning on the basic information and the condition description, and professional initial pre-consultation suggestions are quickly generated, which can improve the accuracy of the pre-consultation. The second language model is then used to combine the basic information and the initial pre-consultation suggestions with the hospital's government affairs information to generate more professional and comprehensive pre-consultation results for the patient, improve the professionalism and effectiveness of the pre-consultation, and enhance the patient experience.

[0138] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0139] The present application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, Figures 2 to 5 The steps of any intelligent pre-diagnosis method are represented.

[0140] The embodiment of the present application also provides an intelligent device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, Figures 2 to 5 The steps of any intelligent pre-diagnosis method are represented.

[0141] The present application also provides a computer program product, which, when executed on a server, enables the server to execute the following Figures 2 to 5 The steps of any intelligent pre-diagnosis method are represented.

[0142] Figure 7 Schematic diagram of a smart device provided by an embodiment of the present application. Figure 7As shown, the smart device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned smart pre-diagnosis method embodiments are implemented, for example Figure 2 Alternatively, when the processor 70 executes the computer program 72, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 6 The functions of units 61 to 64 are shown.

[0143] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the smart device 7.

[0144] The smart device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the smart device 7 and does not constitute a limitation of the smart device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smart device 7 may also include input and output devices, network access devices, buses, etc.

[0145] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0146] The memory 71 may be an internal storage unit of the smart device 7, such as a hard disk or memory of the smart device 7. The memory 71 may also be an external storage device of the smart device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smart device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the smart device 7. The memory 71 is used to store the computer program and other programs and data required by the smart device. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0147] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0148] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0150] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0151] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An intelligent pre-diagnosis method, characterized in that: The intelligent pre-diagnosis method comprises: Obtain basic information of patients; Generate a target digital person based on the basic information, wherein the target digital person is used to interact with the patient according to an interaction strategy to guide the patient to provide a description of the condition; Using the first language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate initial pre-diagnosis suggestions; The basic information and the initial pre-consultation suggestion are combined with hospital government affairs information using a second language model to generate a pre-consultation result for the patient.

2. The intelligent pre-diagnosis method according to claim 1, characterized in that: The basic information includes age and gender. The step of obtaining the basic information of the patient includes: obtaining a facial image of the patient; The facial image is input into a pre-trained multi-task convolutional neural network model, and the multi-task convolutional neural network model is used to identify the age and gender of the patient.

3. The intelligent pre-diagnosis method according to claim 1, characterized in that: The step of using the first language model to semantically understand and infer the basic information and the condition description to generate an initial pre-diagnosis suggestion includes: Fill the basic information and the condition description into a preset prompt template to obtain a prompt; The prompt is input into the first large language model, guiding the large language model to perform semantic understanding and reasoning on the basic information and the condition description, and generating an initial pre-diagnosis suggestion.

4. The intelligent pre-diagnosis method according to any one of claims 1 to 3, characterized in that: The step of generating a target digital human according to the basic information includes: According to the basic information of the patient, selecting an initial digital human model corresponding to the basic information from a predefined digital human model library; determining an interaction style corresponding to the patient; A target digital human is generated based on the interaction style and the initial digital human model.

5. The intelligent pre-diagnosis method according to claim 1, characterized in that: The method further comprises: Identifying the emotional state of the patient during the interaction with the target digital human; According to the emotional state, the interaction strategy of the target digital person and the patient is adjusted.

6. The intelligent pre-diagnosis method according to claim 5, characterized in that: The step of identifying the emotional state of the patient during the interaction with the target digital human comprises: Acquire the facial image of the patient and the voice information of the patient during the interaction process; Inputting the facial image of the patient during the interaction into a facial expression recognition model to recognize the facial emotion of the patient; Inputting the patient's voice information during the interaction into a voice emotion recognition model to identify the patient's voice emotion state; The facial emotion is combined with the speech emotion state to determine the patient's emotional state.

7. The intelligent pre-diagnosis method according to claim 5, characterized in that: The step of adjusting the interaction strategy between the target digital person and the patient according to the emotional state includes: According to the emotional state, at least one of the interaction style, interaction guide sentence, interaction rhythm and interaction animation of the target digital human is adjusted.

8. An intelligent pre-diagnosis device, characterized in that: The intelligent pre-diagnosis device comprises: An information acquisition unit, used to obtain basic information of the patient; An intelligent interaction unit, used to generate a target digital person according to the basic information, wherein the target digital person is used to interact with the patient according to an interaction strategy and guide the patient to provide a description of the condition; An initial suggestion generating unit, configured to use a first language model to perform semantic understanding and reasoning on the basic information and the condition description, and generate an initial pre-diagnosis suggestion; The pre-consultation result generating unit is used to combine the basic information and the initial pre-consultation suggestion with the hospital government affairs information by using the second language model to generate the pre-consultation result of the patient.

9. An intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the intelligent pre-diagnosis method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intelligent pre-diagnosis method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Psychological counseling robot based on multi-modal data and large vertical model

    CN119167308A

  • Psychological consultation robot based on multimodal data and vertical large model

    CN119167308B

  • An artificial intelligence-based emergency patient triage system

    CN119785996A

  • Traditional Chinese medicine intelligent inquiry method and system based on AI big language model

    CN120067279A

  • Traditional Chinese Medicine Intelligent Inquiry Method and System Based on AI Large Language Model

    CN120067279B