Interaction method and device for online inquiry and electronic equipment

Abstract

The invention discloses an online inquiry interaction method and device and electronic equipment, and relates to the technical field of internet medical treatment. A specific embodiment of the method comprises the steps of determining doctor information based on an outpatient medical record picture uploaded by a user, and generating a user online inquiry file; determining a doctor intelligent branch corresponding to the doctor information; displaying a session interface between the user and the doctor to send the user online inquiry file to the session interface, and receiving content input by the user in the session interface; and controlling the doctor intelligent branch to determine reply content according to the input content and the user online inquiry file, displaying the reply content in a session interface, and presenting the reply content by using sound information of the doctor. According to the embodiment, the workload of doctors is relieved, it is also ensured that patients can obtain instant medical advices, and the overall quality and response speed of medical services are improved.

Description

Technical Field

[0001] The present invention relates to the field of Internet medical technology, and in particular to an interactive method, device and electronic equipment for online medical consultation. Background Art

[0002] With the rapid development of Internet medical care, more and more doctors and patients use online platforms to diagnose diseases and renew prescriptions, forming a huge user group and doctor resources. However, facing tens of thousands of patients, manual management by doctors is impractical, and limited by doctors' time and energy, resulting in low efficiency and high cost of existing communication and management methods. Therefore, it is necessary to develop an automated and intelligent online management mechanism to significantly improve the communication efficiency between doctors and patients, improve management methods, and thus improve patient satisfaction and treatment effects. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide an interactive method, device and electronic device for online medical consultation, which can at least solve the problem of low efficiency and high cost of doctor-patient communication and management methods in the prior art.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, an interactive method for online consultation is provided, comprising:

[0005] Based on the outpatient medical record pictures uploaded by the user, determine the doctor's information and generate the user's online consultation file;

[0006] Determine the doctor's intelligent avatar corresponding to the doctor's information;

[0007] Displaying a conversation interface between the user and the doctor, sending the user's online consultation file to the conversation interface, and receiving the content input by the user in the conversation interface;

[0008] Control the doctor's intelligent clone to determine the reply content and display it in the conversation interface based on the input content and the user's online consultation file, and use the doctor's voice information to present the reply content.

[0009] To achieve the above object, according to another aspect of an embodiment of the present invention, an interactive device for online medical consultation is provided, comprising:

[0010] The determination module is used to determine the doctor's information and generate the user's online consultation file based on the outpatient medical record pictures uploaded by the user;

[0011] Determine the doctor's intelligent avatar corresponding to the doctor's information;

[0012] The conversation module is used to display the conversation interface between the user and the doctor, to send the user's online consultation file to the conversation interface, and to receive the content input by the user in the conversation interface;

[0013] The reply module is used to control the doctor's intelligent clone to determine the reply content and display it in the conversation interface based on the input content and the user's online consultation file, and to present the reply content using the doctor's voice information.

[0014] To achieve the above objective, according to another aspect of an embodiment of the present invention, an interactive electronic device for online medical consultation is provided.

[0015] The electronic device of an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned interactive methods for online medical consultation.

[0016] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the above-mentioned interactive methods for online medical consultation is implemented.

[0017] To achieve the above object, according to another aspect of an embodiment of the present invention, a computer program product is provided. A computer program product of an embodiment of the present invention includes a computer program, and when the program is executed by a processor, the interactive method of online consultation provided by an embodiment of the present invention is implemented.

[0018] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: automatically generating a user's online consultation file based on the outpatient medical record pictures uploaded by the user, and matching the corresponding doctor AI clone, ensuring the speed of the consultation process. The doctor's AI clone can respond to the patient's consultation in real time, provide timely and professional replies, and use the doctor's voice information to present the reply content, enhancing the sense of reality and trust. This operation not only reduces the doctor's workload, but also ensures that patients can get immediate medical advice, improving the overall quality and response speed of medical services.

[0019] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0021] Figure 1 This is a schematic diagram of the main flow of an interactive method for online medical consultation according to an embodiment of the present invention;

[0022] Figure 2(a) is a schematic diagram of the reply content;

[0023] Figure 2(b) is a schematic diagram of the interaction between a real doctor, the doctor's AI avatar, and the patient;

[0024] Figure 3 is a flow chart of an optional interactive method for online medical consultation according to an embodiment of the present invention;

[0025] Figure 4 is a flowchart of another optional interactive method for online consultation according to an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the process of creating a doctor's AI clone;

[0027] Figure 6 is a flowchart of another optional interactive method for online consultation according to an embodiment of the present invention;

[0028] Figure 7 It is an interactive diagram of the doctor’s AI avatar making a follow-up plan and the doctor’s remote consultation;

[0029] Figure 8 It is the overall process and architecture diagram of the interactive method of online consultation;

[0030] Fig. 9 This is a schematic diagram of main modules of an interactive device for online medical consultation according to an embodiment of the present invention;

[0031] Fig.10 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0032] Fig.11 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0035] In the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The acquisition, transmission, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations, and are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities.

[0036] For user information, necessary measures should be taken to prevent illegal access to such personal information data, ensure that persons who have access to personal information data comply with relevant laws and regulations, and ensure the security of user personal information. Once such user personal information data is no longer needed, risks should be minimized by limiting or even prohibiting data collection and / or deleting data. When applicable, including in certain relevant applications, protect user privacy by de-identifying data, such as by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the specific address level), controlling how data is stored, and / or other methods of de-identification.

[0037] With the rapid development of Internet medical care, online platforms have attracted nearly one million doctors to join, covering a comprehensive range of departments and diseases, and serving nearly 10 million users for online diagnosis and follow-up prescription renewal. However, in the post-diagnosis scenario, the number of patients served by doctors has accumulated to tens of thousands, and manual management by doctors alone has become impractical. In order to help doctors better manage patients, major Internet medical online platforms support patient tagging, multi-dimensional retrieval, batch reminders and other capabilities. Even so, this communication and management method between doctors and patients still has many limitations, such as low efficiency and high cost of manual operation, limited time and energy of doctors, and rigid management methods.

[0038] A convenient, intelligent and efficient post-diagnosis online patient management system should have the following features: 1. Efficient conversion and standardized archives: Support offline patients to quickly convert to online and form standardized health archives. 2. Automatic binding and real-time communication: Automatically bind doctors according to patient archives, add to the corresponding doctor's management list, and create doctor-patient conversations to communicate about the condition in real time. 3. Professional and friendly responses and guidance: For patients' online consultations, ensure that they can get professional and friendly responses from doctors in a timely manner and obtain guidance and suggestions for the next step. 4. Personalized follow-up plan: The system regularly conducts personalized follow-up for patients and understands the diagnosis and treatment effects according to the plan.

[0039] However, at present, post-diagnosis management mainly relies on manual operation of doctors and traditional medical information systems. The specific methods include: 1. Patients manually fill in and upload information, doctors mark patients in advance, and then communicate with patients by phone and other means based on patient labels to provide post-diagnosis guidance and management. 2. The system regularly sends follow-up scales to patients to collect post-diagnosis information, and then reminds doctors to interpret it. These methods have the following disadvantages:

[0040] 1. Complex process and high patient churn rate: The existing online process for post-diagnosis patients is cumbersome, resulting in patient churn and making it difficult for doctors to manage effectively.

[0041] 2. Untimely response to consultation: Since doctors cannot provide consultations on the online platform 24 hours a day, patients’ consultations cannot be responded to in a timely manner when the doctor is not online.

[0042] 3. Doctors have limited energy: Existing post-diagnosis management relies on manual operations by doctors. Doctors have limited time and energy and are unable to continuously track and manage each patient. This results in patients not receiving timely guidance and help after diagnosis, affecting the treatment effect.

[0043] 4. Lack of intelligence and personalization: Traditional methods cannot provide targeted suggestions and treatment plans based on the patient's specific situation. Although the existing medical information system can record patient information, it cannot perform intelligent analysis and decision-making, and cannot meet the patient's personalized needs.

[0044] In summary, there is an urgent need to develop a smarter and more efficient post-diagnosis online patient management system to address the shortcomings of existing technologies. This can not only significantly improve doctors' work efficiency and reduce management costs, but also optimize the quality of doctor-patient communication and improve patient satisfaction and treatment effects.

[0045] The present invention builds an AI (Artificial Intelligence) avatar for a real doctor through big model technology, and uses artificial intelligence technology to simulate the doctor's consultation process and treatment suggestions, so as to provide patients with continuous medical services and management online in the absence of the doctor.

[0046] The specific steps are as follows:

[0047] 1. Collect doctor information and build AI clones: Collect doctors’ real medical cases, facial and voice information, etc., and build personalized AI clones for each doctor based on these data to ensure that the AI ​​clone can accurately simulate the doctor’s professional behavior and expression style.

[0048] 2. Build medical image recognition capabilities: Develop advanced medical image recognition systems to analyze medical records uploaded by patients (such as outpatient medical record images) in real time, and automatically build online personal files for patients based on the recognition results to ensure the accuracy and timeliness of the file content.

[0049] 3. Match doctors and record conversation relationships: Identify doctor information based on the medical records uploaded by the patient, match the most suitable doctor from the online doctor database, and record the relationship between the doctor and the patient and the content of the conversation online to ensure that each consultation is documented for subsequent tracking and management.

[0050] 4. Personalized responses and continuous health management: Based on the patient's personal profile and conversation intent, the doctor's AI clone can respond to the patient's inquiries in real time, recommend appropriate services, and conduct regular follow-up visits. This not only provides immediate medical advice, but also enables full-process tracking and management of the patient's recovery, ensuring that the patient receives continuous high-quality medical services.

[0051] Through the above steps, this solution not only improves the patient's consultation experience, but also greatly improves the efficiency and quality of medical services through intelligent means, achieving more efficient utilization of medical resources and personalized health management.

[0052] See also Figure 1 , which shows a main flow chart of an interactive method for online consultation provided by an embodiment of the present invention, including the following steps:

[0053] S101: Determine the doctor information based on the outpatient medical record picture uploaded by the user, and generate the user's online consultation file;

[0054] S102: Determine the doctor's intelligent avatar corresponding to the doctor's information;

[0055] S103: Displaying a conversation interface between the user and the doctor, sending the user's online consultation file to the conversation interface, and receiving content input by the user in the conversation interface;

[0056] S104: Control the doctor's intelligent clone to determine the reply content according to the input content and the user's online consultation file and display it in the conversation interface, and use the doctor's voice information to present the reply content.

[0057] In the Internet medical scenario, the entire process of post-diagnosis patient management has at least 8 steps, as follows: 1. Patient offline consultation; 2. Doctor issues outpatient medical records; 3. Patient binds the doctor's online account; 4. Patient submits basic information and outpatient medical record pictures; 5. Doctor receives patient information; 6. Patient consults the doctor about his or her condition online; 7. Doctor replies to the patient online; 8. Doctor follows up with the patient regularly.

[0058] For step S101, an optimization solution is proposed to solve the problem that the operation of step 4 (patients submitting basic information and outpatient medical record pictures) in the existing process is complicated and time-consuming. At present, when submitting information, patients need to fill in their name, age, disease, etc., and upload medical record pictures. The whole process takes at least several minutes on average, resulting in many patients being lost during the submission process.

[0059] In order to simplify the filling of patient information and reduce the loss rate, this solution allows users to quickly log in to the Internet medical system and upload offline consultation medical record pictures through scanning, SMS link or main site search. The system uses large model image understanding and OCR (Optical Character Recognition) technology to automatically identify outpatient medical record pictures, form structured patient data, and automatically generate online files and bind corresponding doctors to establish doctor-patient conversations. In this way, the three steps of patient binding doctor account, submitting information and doctor receiving information (the above steps 3, 4, and 5) that originally needed to be completed separately are combined into one step, which greatly simplifies the patient operation process and effectively reduces the patient loss rate.

[0060] For step S102, based on the above-mentioned doctor-patient information, the system can establish a complete online file for the patient, and match the online doctors on the platform according to the hospital and doctor information (or only the doctor information), and determine the online doctor's AI avatar according to the online doctor's account, so as to establish a complete online doctor-patient relationship. This solution does not consider the expression and posture. The doctor's AI avatar is generated based on the doctor's basic information (such as image, voice, expression style, etc.) and the knowledge base. The knowledge base is established based on the doctor's online diagnosis and treatment data. The diagnosis and treatment data includes one or more of the diagnosis dialogue, consultation records, prescription records, and electronic medical record records, preferably including consultation records, prescription records, and electronic medical record records.

[0061] The key to post-diagnosis management is to give patients the experience of communicating with real doctors. Therefore, this solution is based on the above knowledge base, so that the doctor's personalized style is considered in consultation, prescription, and medical record. This solution uses artificial intelligence and speech synthesis technology to build a virtual intelligent body based on the basic information and knowledge base of the above doctors, simulate the real doctor to complete the process of consultation and consultation, provide patients with continuous medical services and management plans, and ensure that patients receive efficient and personalized medical support.

[0062] For step S103, a doctor-patient conversation is established, specifically, the conversation interface between the user and the doctor is displayed. In order to optimize the process of establishing a doctor-patient conversation, especially in the online follow-up scenario after an offline consultation, this solution is designed as follows: When a user visits a doctor for the first time, since an online binding relationship has not yet been established between the patient and the doctor, the patient's account information does not exist in the doctor's online user management list. The system will guide the patient to scan the code, upload medical record pictures, and create a new patient file before binding the doctor. Once the binding is successful, the first online conversation can begin. For users who already have a binding relationship, during subsequent multiple visits, the patient's account information already exists in the doctor's online user management list, so the previously created conversation box can be directly opened and communication can be carried out, which simplifies the user's operation process and improves communication efficiency.

[0063] For step S104, in steps 6 and 7 of the existing process, the patient consults the doctor online about his condition. After sending a message in the conversation box, the doctor is not online, so the doctor cannot reply in time, which will make the patient experience poor and even affect the treatment time. In order to improve the patient's online consultation experience and reduce the waiting time caused by the doctor's offline, this solution builds an AI clone for the doctor, so that the doctor's AI clone can not only understand and handle the patient's consultation when replying to the patient's questions, but also provide personalized diagnosis and treatment suggestions and services based on historical data, so that the patient can get real-time professional answers and feel the same consultation effect as a real doctor. The diagnosis and treatment suggestions here refer to precautions, such as drinking more water, exercising more, controlling diet, etc. These are not diagnoses, and the specific diagnosis requires a real doctor to confirm.

[0064] When a patient initiates a consultation, the doctor's AI clone can identify the user's intention based on the patient's input content and online medical records (such as medical records, drug information in diagnostic reports), through the natural language processing model, to determine whether the "keywords for contacting the doctor" are hit, and then determine whether it is necessary to contact the doctor. As shown in Figure 2(a), for the question "What causes headaches?" initiated by the user, the doctor's AI clone can respond based on the doctor's knowledge base, so that the reply content is consistent with the doctor's expression style, such as "The causes of headaches are complex, including dietary, endocrine, mental and other factors, environmental stimulation, emotional instability can also cause headaches. Intracranial lesions and other organ diseases can also cause headaches."

[0065] The doctor's AI clone can simulate the doctor's voice, so when presenting the recovery content, the doctor's voice can be used to verbally express the reply content. Or, specific options can be set at specific locations such as the side of the reply content or the side of the conversation box, such as a speaker button. Only when the patient clicks the button, the doctor's AI clone will use the doctor's voice to express the reply content.

[0066] In addition, the doctor's AI clone also has a recommendation function, which can recommend the doctor's service items according to the patient's needs, such as the consultation service shown in Figure 2(a). However, it should be noted that the premise for the recommendation of this service is that the doctor is online. If the doctor is not online, the system will not recommend the service to the patient, so as to avoid the patient not being able to get the doctor's professional help in time after paying. If the patient clicks on the relevant service link, the system will automatically notify the doctor himself to ensure that the patient can get the professional guidance of the real doctor in time when needed. This design not only improves the immediacy of patient consultation, but also provides patients with clear operation instructions, avoiding the anxiety caused by long waiting times.

[0067] It should be noted that when the doctor is not online (i.e. the first preset condition) or the doctor is online but the patient has not paid (i.e. the second preset condition), the doctor's AI clone can intervene first to provide the patient with basic diagnosis and treatment advice and services, but when it comes to specific diagnosis, the patient still needs to be guided to seek help from a real doctor, so the overall interaction structure is shown in Figure 2(b). This is because communication with a real online doctor requires payment, and the doctor will be paid a commission after the consultation is completed, so not all replies are directly made through the doctor's AI clone. This method not only ensures the professionalism and rigor of medical services, but also reflects respect for the value of doctors' labor.

[0068] There are many specific ways to determine whether a doctor is online. For example, the system sets an online status indicator for each doctor. When the doctor logs in, the system sets the indicator to "online", otherwise, it is set to "offline" when the doctor logs out. Or if the doctor has not operated the doctor's terminal (such as a mobile phone or computer) for a long time, the doctor's status is set to "away". Or the doctor changes his status to "resting". Whether offline, away or resting, it is not online. In this way, the doctor's online status can be tracked in real time.

[0069] Furthermore, real-time communication technologies such as WebSocket can be used to maintain a long connection between the doctor and the server. The connection is established when the doctor is online and disconnected when the doctor is offline, ensuring that the doctor's online status can be updated and perceived in real time.

[0070] Some doctors have working hours, so doctors can set the time periods for consultation in the system in advance according to their work schedules, and clearly mark them as online time periods. When they are not in these preset time periods, they are offline by default. This approach helps to reasonably plan the doctor's working hours, and also makes it easier for patients to choose a suitable consultation time. The above technical features can be used in combination or separately, without specific restrictions.

[0071] The specific method for judging whether the patient has not paid is to take the consultation service shown in Figure 2(a) as an example. The patient clicks on the consultation service and performs the payment operation. The system queries the patient's order status based on the patient's account information. If the order status is displayed as "paid", it is considered that the patient has completed the payment; otherwise, if the order status is "unpaid" or "payment failed", the next step is not allowed. However, if the patient does not click on the consultation service within a certain period of time (for example, 5 minutes) and does not perform the payment operation, the doctor's AI clone needs to intervene.

[0072] Sometimes payment needs to be completed through a third-party payment platform (such as a bank). When the patient completes the payment through the third-party payment platform, the third-party payment platform will send a callback notification of successful payment to the system. After receiving this notification, the system updates the patient's relevant records to confirm that the payment has been made. For possible delays in certain payment methods, the system can regularly call the query interface provided by the payment platform to actively obtain the latest payment results to ensure timely and accurate understanding of the patient's payment status.

[0073] At the same time, the doctor's AI avatar is similar to the doctor's real account. It can display the doctor's head portrait in the general IM (Instant Messaging) conversation and communicate with patients through text, voice, etc., instead of the digital human display form, which is not compatible with the traditional IM method. This operation ensures the consistency and coherence of the user experience, allowing users to feel the experience of communicating with real doctors during the post-diagnosis management process.

[0074] The method provided in the above embodiment significantly improves the efficiency and experience of patient consultations through automation and intelligent means. Specifically, the user's online consultation file is automatically generated based on the outpatient medical record pictures uploaded by the user, and the corresponding doctor AI clone is matched to ensure the personalization and professionalism of the consultation process. The doctor's AI clone can respond to patients' inquiries in real time, provide timely and professional replies, and use the doctor's voice information to present the reply content, enhancing the sense of reality and trust. This operation not only reduces the doctor's workload, but also ensures that patients can get immediate medical advice, improving the overall quality and response speed of medical services.

[0075] See also Figure 3 , shows a schematic flow chart of an optional interactive method for online consultation according to an embodiment of the present invention, comprising the following steps:

[0076] S301: using large model image understanding technology and character recognition technology, based on pre-set doctor prompt words, user prompt words and diagnosis prompt words, identifying doctor information, user information and diagnosis information from the outpatient medical record picture;

[0077] S302: Generate a user online consultation file based on user information, diagnosis information and outpatient medical record images.

[0078] In the above implementation, for step S301, the core capability of this solution process is to accurately identify the outpatient medical record pictures provided by the user. By adopting large-model image understanding technology and combining professional and detailed prompt word guidance, it is ensured that the large model can accurately extract important data such as patient information, doctor information, hospital information and diagnosis results through prompt words. The large model refers to the artificial intelligence large model, which is a machine learning model with ultra-large-scale parameters (usually more than one billion) and super computing resources. It can process massive data and complete various complex tasks, such as natural language processing. The recognition process follows the following rules:

[0079] 1. Input format: To ensure the accuracy of recognition, the input image should be in JPEG or PNG format with a resolution of no less than 300dpi. In addition, the system will also apply medical image preprocessing technology, including adjusting contrast, removing noise and correcting tilt, to optimize the quality of medical record images, thereby further improving recognition accuracy.

[0080] 2. Identify content: Extract the following information from the image:

[0081] - Patient information (such as name, age, gender)

[0082] - Doctor's information (such as name, title)

[0083] - Hospital information (such as hospital name, department)

[0084] -Diagnosis (e.g., diagnosis name, recommendations)

[0085] 3. Output format: Output all recognized information in JSON format. The example is as follows:

[0086]

[0087] In addition to the above prompt words, this solution can also set other prompt words, such as medication information and time information. Time information can facilitate subsequent doctor AI clones to monitor changes in user files. For example:

[0088]

[0089]

[0090] For step S302, based on user information, diagnosis information and outpatient medical record pictures, the system will automatically generate an online consultation file for the user. Each user can have multiple online consultation files, depending on their medical treatment. For example, if a user visits the dermatology department and gynecology department on the same day, and the doctors in these two departments are different, two independent online consultation files will be generated. Similarly, even if a user visits the same doctor multiple times, as long as the visit times are different, different online consultation files will be generated. For example, if a user visited the same dermatology doctor yesterday and today, the system will generate an independent online consultation file for each visit to ensure the completeness and accuracy of each diagnosis and treatment record.

[0091] The method provided in the above embodiment uses large-model image understanding technology and character recognition technology, combined with pre-set doctor prompt words, user prompt words and diagnosis prompt words, to accurately identify doctor information, user information and diagnosis information from outpatient medical record pictures, and generate user online consultation files based on this information. This operation can greatly improve the accuracy and efficiency of information extraction, reduce the cumbersome steps and error rate of manual input, and quickly generate structured online consultation files to ensure the integrity and accuracy of each consultation record, thereby improving patient experience and reducing churn.

[0092] See also Figure 4 , shows a schematic flow chart of another optional interactive method for online consultation according to an embodiment of the present invention, comprising the following steps:

[0093] S401: Processing the doctor's photo by constructing an avatar to obtain the doctor's avatar;

[0094] S402: Processing the doctor's voice file through a preset speech synthesis model to obtain the doctor's voice;

[0095] S403: Through the big model, the doctor's portrait, the doctor's voice, and the knowledge base are processed to obtain the doctor's intelligent avatar, and then the association between the doctor's intelligent avatar and the doctor's account information is established.

[0096] In the above implementation, the establishment of the doctor's AI avatar includes the establishment of voice, image, and knowledge base, which respectively include a personalized speech synthesis model, a graphic AI avatar image, and an online real medical record knowledge base. For the overall flow chart, see Figure 5 As shown:

[0097] For step S401, the image of the doctor is mainly realized by constructing an AI avatar. This solution takes into account cost and efficiency and uses the OpenCV (Open Source Computer Vision Library) library to generate a cartoon-style AI avatar. To distinguish it from the real doctor's avatar, the generated AI avatar is in a cartoon style, retaining the basic contour features of the real doctor's hairstyle, face shape, etc., making the avatar look smoother, cartoon-like and digital. Among them, OpenCV is an open source computer vision and machine learning software library that provides a wealth of image processing and computer vision algorithms, and is widely used in real-time image processing, video analysis, object recognition and other fields.

[0098] The generation process is mainly divided into the following 7 steps to convert the real image of the doctor into a cartoon style:

[0099] 1. Preprocess the doctor's photo and convert it to grayscale to reduce complexity. Before converting to grayscale, you can also resize the image, such as using cv2.resize() to resize the image, which is used to adjust the width and height of the image.

[0100] 2. Perform edge detection on the grayscale image, generate a binary edge map, and extract the contour. Here you can use cv2.adaptiveThreshold() for edge detection. This function is used to apply adaptive threshold processing to the image. The input image (must be a single channel, that is, a grayscale image) is used. Unlike the global threshold, the adaptive threshold can dynamically adjust the threshold according to different areas of the image, so as to better handle situations such as uneven lighting.

[0101] 3. To obtain a clear cartoon effect, apply bilateral filtering to the original image for smoothing to reduce noise, such as removing details such as facial wrinkles and acne marks. Among them, bilateral filtering, such as cv2.bilateralFilter(), is a nonlinear filtering method that can smooth the image while retaining edge information. It combines spatial proximity and pixel intensity similarity for filtering.

[0102] 4. Convert the original doctor's photo to HSV (Hue, Saturation, Value) color space and create a skin color mask to highlight the doctor's facial features. The purpose of this operation is to obtain the doctor's real skin color. Here, you can use cv2.inRange() to create a skin color mask. This function is used to create a binary mask to mark pixels in the image within a specified range.

[0103] 5. Fuse the smoothed image obtained in step 3 with the skin color mask obtained in step 4 to obtain a cartoonized foreground, and combine it with the edge map to highlight the contour. Here, cv2.bitwise_and() can be used to fuse the skin color with the smoothed image.

[0104] 6. Generate a suitable background color, such as light green, or directly set the general background color. Use np.full() to create a background image of the same size as the avatar.

[0105] 7. Synthesize the cartoon avatar with the background image and save the final cartoon avatar image. Here, you can use cv2.add() to synthesize the cartoon avatar with the background, and use cv2.imwrite() to save the final cartoon avatar image.

[0106] For step S402, the doctor's personalized speech synthesis model is completed through TensorFlow's FastSpeech implementation TensorFlowTTS, such as U-Net. The doctor's voice files, telephone audio files, and welcome settings and other voice files on the online platform are obtained in advance, and these voice files are used for training to form a personalized voice model. When the doctor's AI avatar replies to the patient, the model can use the doctor's voice to convert the text content of the reply into speech, which is convenient for the patient to understand.

[0107] For step S403, the system automatically collects various data generated by doctors during the actual online diagnosis and treatment process, including online consultation records, prescription records, and electronic medical records, etc., or one or more of them, preferably all data. These data will be desensitized after collection to protect the privacy of patients. The desensitized data will be stored in the vector database to ensure efficient data management and rapid retrieval. The doctor's knowledge base will be continuously updated, because the doctor's diagnosis and treatment related data is continuously accumulated, and the information in the knowledge base also needs to be updated accordingly. By collecting data from real doctors, the reply effect of the doctor's AI clone can be optimized to make it closer to the expression style of real doctors.

[0108] Based on the established doctor AI image, doctor AI voice and doctor AI knowledge base, a large model is used for processing to generate a complete doctor AI clone and establish an association between the doctor account and the doctor AI clone. Through the doctor AI clone's image and voice similar to that of a real doctor and its professional and friendly service methods, the system completes the establishment of the doctor AI clone, ensuring that it highly restores the diagnosis and treatment experience of a real doctor.

[0109] The method provided in the above embodiment collects the doctor's real medical treatment cases, facial and voice information, and uses large model technology to build an AI avatar for the doctor. The doctor's AI avatar has an image, voice, and professional and friendly service style that are similar to those of the real doctor, ensuring that the diagnosis and treatment experience of the real doctor is highly restored.

[0110] See also Figure 6 , shows a flow chart of another optional interactive method for online consultation according to an embodiment of the present invention, comprising the following steps:

[0111] S601: Control the doctor's smart avatar to determine the online user management list corresponding to the doctor's account information, and monitor the changes of the online consultation files associated with the doctor under each user's account information according to the account information of each user in the online user management list;

[0112] S602: In response to receiving consultation information input by the user in the conversation interface, or monitoring changes in the user's online consultation file, analyzing the user's relevant information to formulate a follow-up plan and send it to the user through the conversation interface, or triggering an early warning mechanism to notify the doctor.

[0113] In the above implementation, in the existing process, in step 8, doctors follow up patients regularly. Facing tens of thousands of patients, doctors cannot follow up one by one. In order to solve the huge workload problem faced by doctors when regularly following up tens of thousands of patients, see Figure 7 As shown in the figure, this solution introduces a doctor AI clone to achieve full-process automated follow-up of patients without the need for doctors to manually manage the patient. By analyzing and understanding the patient's condition and demands in real time, the doctor's AI clone can automatically respond to patient inquiries, recommend appropriate online doctor services, and support regular follow-up and long-term tracking and analysis of the patient's condition to ensure that the patient receives continuous attention and professional medical advice.

[0114] The doctor's AI clone can not only automatically generate and send a personalized follow-up plan based on the patient's consultation or file changes, but also summarize the patient's condition information and synchronize it to the doctor, integrate the risk management mechanism, and promptly remind the doctor to intervene when the patient's condition is at high risk, so as to realize remote consultation and remote diagnosis and prescription. This not only reduces the doctor's workload, but also ensures that the patient receives timely and effective guidance and support throughout the recovery process.

[0115] For example, when a patient tells the doctor online that "I was discharged today", the doctor's AI clone will immediately reply to the patient's notes and automatically formulate a 15-day rehabilitation follow-up plan for him. During the 15-day follow-up, the AI ​​clone will regularly remind the patient to take medication on time and report the recovery status to ensure the smooth progress of the rehabilitation process. For another example, when a patient updates the prescription in the file, the doctor's AI clone will send the patient a medication compliance follow-up plan based on the new prescription information, encourage the patient to report the medication status on time, and improve the patient's participation and compliance through a reward mechanism.

[0116] The method provided in the above embodiment, the doctor's AI clone can help the doctor automatically formulate and send follow-up plans, track the patient's recovery status throughout the entire process, and ensure that each patient can receive timely and professional guidance and support during the recovery process. At the same time, it effectively reduces the doctor's work pressure and improves the overall efficiency and quality of medical services.

[0117] See also Figure 8 , showing the overall process and architecture of the interactive method of online consultation. This solution aims to simplify the patient operation process by using intelligent technology, complete the establishment of patient files and the binding relationship with doctors with one click; respond to patients' consultation questions in real time, automatically identify patients' demands and help doctors reply to patients; continue to pay attention to the patient's diagnosis and treatment process, and dynamically manage patients through automatic follow-up and scale services, so as to improve the efficiency of medical services, patient satisfaction and treatment effects.

[0118] 1. Improve the efficiency of medical services: Through AI avatars, patients' consultation needs can be responded to quickly, reducing the workload of doctors, thereby greatly improving the efficiency of overall medical services.

[0119] 2. Personalized medical services: With the help of medical image recognition technology and real-time construction of patient personal files, personalized diagnosis and treatment advice and services are provided to each patient, significantly improving patient satisfaction and treatment effects.

[0120] 3. Continuous health management: Through regular follow-up and the recommendation of adaptive services, patients can receive continuous health management to prevent the recurrence and deterioration of the disease, thereby improving the overall health level of patients.

[0121] Therefore, this solution not only optimizes the patient's consultation experience, but also improves the quality and efficiency of medical services through intelligent means, achieving more efficient utilization of medical resources and personalized health management.

[0122] See also Fig. 9 , showing a schematic diagram of main modules of an interactive device 900 for online medical consultation provided by an embodiment of the present invention, including:

[0123] The determination module 901 is used to determine the doctor information and generate the user's online consultation file based on the outpatient medical record picture uploaded by the user;

[0124] Determine the doctor's intelligent avatar corresponding to the doctor's information;

[0125] The conversation module 902 is used to display the conversation interface between the user and the doctor, to send the user's online consultation file to the conversation interface, and to receive the content input by the user in the conversation interface;

[0126] The reply module 903 is used to control the doctor's intelligent clone to determine the reply content and display it in the conversation interface based on the input content and the user's online consultation file, and to present the reply content using the doctor's voice information.

[0127] In the implementation device of the present invention, the determination module 901 is used to:

[0128] Using large model image understanding technology and character recognition technology, based on pre-set doctor prompt words, user prompt words and diagnosis prompt words, identify doctor information, user information and diagnosis information from the outpatient medical record image;

[0129] Generate user online consultation files based on user information, diagnosis information and outpatient medical record images.

[0130] In the implementation device of the present invention, the doctor's AI clone is generated based on the doctor's basic information and knowledge base, and the knowledge base is established based on the doctor's online diagnosis and treatment data.

[0131] In the implementation device of the present invention, the basic information of the doctor includes the doctor's photo and the doctor's voice file. The device also includes a generation module for generating an intelligent avatar of the doctor. The process includes:

[0132] By constructing an avatar, the doctor's photo is processed to obtain the doctor's avatar;

[0133] By using the preset speech synthesis model, the doctor's voice file is processed to obtain the doctor's voice;

[0134] Through the big model, the doctor's portrait, voice, and knowledge base are processed to obtain the doctor's intelligent avatar, and then the association between the doctor's intelligent avatar and the doctor's account information is established.

[0135] In the implementation device of the present invention, the generation module is used to:

[0136] The skin color mask and smoothing operations are performed on the doctor's photo respectively, and the skin color mask and the smoothed image are fused to obtain a cartoon foreground image;

[0137] Extract the edge map of the doctor's photo; fuse the edge map and the foreground map to obtain a cartoonized avatar;

[0138] The cartoonized image and the preset background image are synthesized to obtain the doctor's portrait.

[0139] In the implementation device of the present invention, the generation module is used to:

[0140] Convert doctor photos to grayscale;

[0141] Perform edge detection on the grayscale image to obtain a binary edge image;

[0142] Perform contour extraction on the binary edge map to obtain an edge map.

[0143] In the implementation device of the present invention, the generation module is used to:

[0144] Converting the doctor's photo into a preset color space to obtain a color image, so as to create a skin color mask based on the color image;

[0145] The doctor's photo is smoothed to obtain a smoothed image.

[0146] In the implementation device of the present invention, the online medical treatment data includes one or more of consultation records, prescription records and electronic medical records, and the generation module is also used to: desensitize the online medical treatment data.

[0147] In the implementation device of the present invention, the conversation module 902 is used to:

[0148] Determine an online user management list corresponding to the doctor's account information;

[0149] In response to the user's account information not existing in the online user management list, adding the user's account information to the online user management list; and establishing a conversation box based on the user's account information and the doctor's account information, and sending the user's online consultation file to the conversation interface of the conversation box;

[0150] In response to the user's account information existing in the online user management list, a conversation box corresponding to the user's account information and the doctor's account information is displayed, and the user's online consultation file is sent to the conversation interface of the conversation box.

[0151] In the implementation device of the present invention, the reply module 903 is used to:

[0152] In response to the doctor's current status meeting the first preset condition, or the doctor's current status not meeting the first preset condition but the user's operation meeting the second preset condition, the doctor's intelligent clone is controlled to determine the reply content based on the input content and the user's online consultation file and display it in the conversation interface.

[0153] In the implementation device of the present invention, the process of determining whether the current state of the doctor meets the first preset condition in the reply module 903 includes one or more of the following methods:

[0154] Acquire a consultable time period pre-configured for the doctor, and in response to the current time not being within the consultable time period, determine that the doctor's current state meets a first preset condition;

[0155] Determine the doctor end logged in by the doctor's account information, and in response to the connection between the doctor end and the doctor end being in a disconnected state, determine that the doctor's current state meets the first preset condition;

[0156] The online status of the doctor's account information is obtained, and in response to the online status being offline, it is determined that the doctor's current status meets the first preset condition.

[0157] In the implementation device of the present invention, the process of determining whether the user operation meets the second preset condition in the reply module 903 includes the following methods:

[0158] In response to the user's selection operation of the payment option, calling the payment interface corresponding to the payment option to receive the user's payment information, and in the case where the payment result is not paid or the payment fails, determining that the user operation meets the second preset condition;

[0159] In response to the user not selecting an operation on a payment option within a preset time period, it is determined that the user operation meets a second preset condition.

[0160] In the implementation device of the present invention, the reply module 903 is used to:

[0161] In the process of displaying the reply content, or in response to a user clicking on a preset option, the reply content is presented using the doctor's voice information.

[0162] The implementation device of the present invention also includes a monitoring module, which is used to:

[0163] Control the doctor's smart avatar, determine the online user management list corresponding to the doctor's account information, and monitor the changes in the online consultation files associated with the doctor under each user's account information based on the account information of each user in the online user management list;

[0164] In response to receiving consultation information input by the user in the conversation interface, or monitoring changes in the user's online consultation file, analyze the user's relevant information to formulate a follow-up plan and send it to the user through the conversation interface, or trigger an early warning mechanism to notify the doctor.

[0165] In addition, the specific implementation content of the device described in the embodiment of the present invention has been described in detail in the method described above, so the repeated content will not be described again here.

[0166] Fig.10 An exemplary system architecture 1000 to which embodiments of the present invention may be applied is shown, including terminal devices 1001 , 1002 , 1003 , a network 1004 , and a server 1005 (only an example).

[0167] Terminal devices 1001, 1002, 1003 can be various electronic devices with display screens and supporting web browsing, and various communication client applications are installed. Users can use terminal devices 1001, 1002, 1003 to interact with server 1005 through network 1004 to receive or send messages, etc.

[0168] The network 1004 is used to provide a medium for communication links between the terminal devices 1001, 1002, 1003 and the server 1005. The network 1004 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0169] The server 1005 may be a server that provides various services, such as a backend management server that provides support for shopping websites browsed by users using terminal devices 1001, 1002, and 1003 (for example only). The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal device. It should be noted that the method provided in the embodiment of the present invention is generally executed by the server 1005, and accordingly, the device is generally set in the server 1005.

[0170] It should be understood that Fig.10 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0171] Reference below Fig.11 , which shows a schematic diagram of the structure of a computer system 1100 of a terminal device suitable for implementing an embodiment of the present invention. Fig.11 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0172] like Fig.11As shown, the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the system 1100 are also stored. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0173] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0174] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, the above-mentioned functions defined in the system of the present invention are executed.

[0175] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0176] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a determination module, a conversation module, and a reply module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the conversation module may also be described as a "conversation content module."

[0178] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes any of the above online consultation interactive methods.

[0179] The computer program product of the present invention comprises a computer program, and when the computer program is executed by a processor, the interactive method for online medical consultation in the embodiment of the present invention is implemented.

[0180] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An interactive method for online consultation, characterized in that: include: Based on the outpatient medical record pictures uploaded by the user, determine the doctor's information and generate the user's online consultation file; Determine the doctor's intelligent avatar corresponding to the doctor's information; Displaying a conversation interface between the user and the doctor, sending the user's online consultation file to the conversation interface, and receiving the content input by the user in the conversation interface; Control the doctor's intelligent clone to determine the reply content and display it in the conversation interface based on the input content and the user's online consultation file, and use the doctor's voice information to present the reply content.

2. The method according to claim 1, characterized in that Determining doctor information based on the outpatient medical record pictures uploaded by the user, and generating the user's online consultation file, includes: Using large model image understanding technology and character recognition technology, based on pre-set doctor prompt words, user prompt words and diagnosis prompt words, identify doctor information, user information and diagnosis information from the outpatient medical record image; Generate user online consultation files based on user information, diagnosis information and outpatient medical record images.

3. The method according to claim 1, characterized in that: The doctor's intelligent clone is generated based on the doctor's basic information and knowledge base, and the knowledge base is established based on the doctor's online diagnosis and treatment data.

4. The method according to claim 3, characterized in that The doctor's basic information includes the doctor's photo and voice file. The process of generating the doctor's intelligent avatar includes: By constructing an avatar, the doctor's photo is processed to obtain the doctor's avatar; By using the preset speech synthesis model, the doctor's voice file is processed to obtain the doctor's voice; Through the big model, the doctor's portrait, voice, and knowledge base are processed to obtain the doctor's intelligent avatar, and then the association between the doctor's intelligent avatar and the doctor's account information is established.

5. The method according to claim 4, characterized in that The method of constructing an avatar and processing the doctor's photo to obtain the doctor's avatar includes: The skin color mask and smoothing operations are performed on the doctor's photo respectively, and the skin color mask and the smoothed image are fused to obtain a cartoon foreground image; Extract the edge map of the doctor's photo; fuse the edge map and the foreground map to obtain a cartoon-like avatar; The cartoonized image and the preset background image are synthesized to obtain the doctor's portrait.

6. The method according to claim 5, characterized in that The step of extracting the edge map of the doctor's photo includes: Convert doctor photos to grayscale; Perform edge detection on the grayscale image to obtain a binary edge image; Perform contour extraction on the binary edge map to obtain an edge map.

7. The method according to claim 5, characterized in that The extracting skin color mask and smoothing operations are respectively performed on the doctor's photo, including: Converting the doctor's photo into a preset color space to obtain a color image, so as to create a skin color mask based on the color image; The doctor's photo is smoothed to obtain a smoothed image.

8. The method according to claim 3, characterized in that The online medical treatment data includes one or more of consultation records, prescription records and electronic medical records. The method also includes: desensitizing the online medical treatment data.

9. The method according to claim 1, characterized in that: The display of the conversation interface between the user and the doctor to send the user's online consultation file to the conversation interface includes: Determine an online user management list corresponding to the doctor's account information; In response to the user's account information not existing in the online user management list, adding the user's account information to the online user management list; and establishing a conversation box based on the user's account information and the doctor's account information, and sending the user's online consultation file to the conversation interface of the conversation box; In response to the user's account information existing in the online user management list, a conversation box corresponding to the user's account information and the doctor's account information is displayed, and the user's online consultation file is sent to the conversation interface of the conversation box.

10. The method according to claim 1, characterized in that The control doctor's smart clone determines the reply content and displays it in the conversation interface according to the input content and the user's online consultation file, including: In response to the doctor's current status meeting the first preset condition, or the doctor's current status not meeting the first preset condition but the user's operation meeting the second preset condition, the doctor's intelligent clone is controlled to determine the reply content based on the input content and the user's online consultation file and display it in the conversation interface.

11. The method according to claim 10, characterized in that The process of determining whether the current state of the doctor meets the first preset condition includes one or more of the following methods: Acquire a consultable time period pre-configured for the doctor, and in response to the current time not being within the consultable time period, determine that the doctor's current state meets a first preset condition; Determine the doctor end logged in by the doctor's account information, and in response to the connection between the doctor end and the doctor end being in a disconnected state, determine that the doctor's current state meets the first preset condition; The online status of the doctor's account information is obtained, and in response to the online status being offline, it is determined that the doctor's current status meets the first preset condition.

12. The method according to claim 10 or 11, characterized in that: The process of determining whether the user operation meets the second preset condition includes one of the following methods: In response to the user's selection operation of the payment option, calling the payment interface corresponding to the payment option to receive the user's payment information, and in the case where the payment result is not paid or the payment fails, determining that the user operation meets the second preset condition; In response to the user not selecting an operation on a payment option within a preset time period, it is determined that the user operation meets a second preset condition.

13. The method according to claim 1, characterized in that The presenting of the reply content using the doctor's voice information includes: In the process of displaying the reply content, or in response to a user clicking on a preset option, the reply content is presented using the doctor's voice information.

14. The method according to claim 1, characterized in that The method further comprises: Control the doctor's smart avatar, determine the online user management list corresponding to the doctor's account information, and monitor the changes in the online consultation files associated with the doctor under each user's account information based on the account information of each user in the online user management list; In response to receiving consultation information input by the user in the conversation interface, or monitoring changes in the user's online consultation file, analyze the user's relevant information to formulate a follow-up plan and send it to the user through the conversation interface, or trigger an early warning mechanism to notify the doctor.

15. An interactive device for online medical consultation, characterized in that: include: The determination module is used to determine the doctor's information and generate the user's online consultation file based on the outpatient medical record pictures uploaded by the user; Determine the doctor's intelligent avatar corresponding to the doctor's information; The conversation module is used to display the conversation interface between the user and the doctor, to send the user's online consultation file to the conversation interface, and to receive the content input by the user in the conversation interface; The reply module is used to control the doctor's intelligent clone to determine the reply content and display it in the conversation interface based on the input content and the user's online consultation file, and to present the reply content using the doctor's voice information.

16. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 14.

17. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images