AI standardized patient real-time interaction method and system based on pre-caching

Abstract

The embodiment of the invention provides an AI standardized patient real-time interaction method and system based on pre-caching. The method comprises the following steps: firstly, splitting a question and answer pair consisting of a doctor question and a patient answer from a standard inquiry dialogue, generating digital human materials containing the doctor question and a corresponding digital human material storage path, and storing the digital human materials in a vector database; when a target question of a user is received, performing semantic matching in the vector database, directly outputting a target digital human material corresponding to the target question when the matching is successful, taking original case data of a current patient as a plug-in knowledge base when the matching is unsuccessful, and generating the digital human material as an answer in real time by utilizing a multi-modal large model; and storing the doctor problem and the generated digital human material into a vector database. According to the interaction mechanism, in actual interaction, real-time full-link calculation of a multi-mode large model does not need to be repeatedly triggered, large model real-time reasoning is started only when a new problem is not hit, and a result is supplemented to a cache, so that doctor-patient interaction efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a pre-cached AI standardized patient real-time interaction method and system. Background Art

[0002] With the acceleration of the digitalization of medical education, AI standardized patients have been widely used in medical education, clinical diagnosis and evaluation, medical skills training, and doctor-patient communication training due to their advantages of low cost, high consistency, scalability, and reusability.

[0003] At present, AI standardized patient interactions in traditional modes are highly dependent on real-time full-link computing of large models. Each interaction must complete serial processing links such as text semantic understanding, multimodal data generation, and audio and video synchronization alignment in sequence, so that each interaction requires starting the large model computing link from scratch.

[0004] However, in large-scale medical teaching scenarios, the traditional model relies on real-time full-link computing of large models, which can easily cause slow system response or even crash, resulting in low efficiency of doctor-patient interaction. Summary of the Invention

[0005] The embodiments of the present invention provide a pre-caching-based AI standardized patient real-time interaction method and system, which can improve the efficiency of doctor-patient interaction.

[0006] In a first aspect, an embodiment of the present invention provides a pre-cached AI standardized patient real-time interaction method, the method comprising:

[0007] A1: Obtain at least one question-answer pair based on a standard medical consultation dialogue, use a multimodal large model to generate a digital human material corresponding to each question-answer pair, and store it in a vector database to build a pre-cached knowledge base. Each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path;

[0008] A2: upon receiving a target question input by a user, performing a semantic matching operation on the target question in the vector database, and determining whether the target question matches a preset question in the vector database; if so, executing step A3; otherwise, executing step A4;

[0009] A3: Output the target digital human material that matches the target problem;

[0010] A4: Use the original case data of the current patient as an external knowledge base, utilize the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question for the current question-answer pair, and the text answer as the patient's answer for the current question-answer pair, and return to execute step A1.

[0011] Preferably,

[0012] Said A1 includes:

[0013] Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer;

[0014] Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model;

[0015] Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path;

[0016] The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

[0017] Preferably,

[0018] The step A2 of performing a semantic matching operation on the target question in the vector database includes:

[0019] Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively;

[0020] Determine the cosine similarity between the target question vector and the preset question vector using a first formula;

[0021] The first formula is:

[0022] Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

[0023] Preferably,

[0024] The step A2 of determining whether the target question matches a preset question in the vector database includes:

[0025] Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order;

[0026] Determine whether the confidence of the first ranked item exceeds a preset confidence threshold. If so, execute step A3; otherwise, execute step A4.

[0027] In a second aspect, an embodiment of the present invention provides a pre-cached AI standardized patient real-time interaction system, the system comprising:

[0028] A knowledge base construction module is used to obtain at least one question-answer pair based on a standard medical consultation dialogue, generate a digital human material corresponding to each question-answer pair using a multimodal large model, and store it in a vector database to construct a pre-cached knowledge base. Each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path;

[0029] Semantic matching module: configured to, upon receiving a target question input by a user, perform a semantic matching operation on the target question in the vector database and determine whether the target question matches a preset question in the vector database; if so, trigger the output module; otherwise, execute the trigger processing module;

[0030] The output module is configured to output target digital human materials matching the target question when triggered;

[0031] The processing module is used to, when triggered, use the original case data of the current patient as an external knowledge base, use the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question of the current question and answer pair, and the text answer as the patient's answer to the current question and answer pair, and return to execute the triggering knowledge base construction module.

[0032] Preferably,

[0033] The knowledge base construction module is used to perform:

[0034] Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer;

[0035] Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model;

[0036] Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path;

[0037] The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

[0038] Preferably,

[0039] The semantic matching module is further configured to, upon receiving a target question input by a user, execute:

[0040] Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively;

[0041] Determine the cosine similarity between the target question vector and the preset question vector using a first formula;

[0042] The first formula is:

[0043] Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

[0044] Preferably,

[0045] The semantic matching module is further configured to, after determining the cosine similarity between the target question vector and the preset question vector using the first formula, execute:

[0046] Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order;

[0047] Determine whether the confidence of the first ranked item exceeds a preset confidence threshold. If so, trigger the output module; otherwise, execute the trigger processing module.

[0048] In a third aspect, an embodiment of the present invention provides a pre-caching-based AI standardized patient real-time interaction system, comprising: at least one memory and at least one processor;

[0049] The at least one memory is configured to store a machine-readable program;

[0050] The at least one processor is configured to call the machine-readable program to execute any one of the methods described in the first aspect.

[0051] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the processor is caused to execute any of the methods described in the first aspect.

[0052] The embodiment of the present invention provides a method and system for real-time interaction with AI standardized patients based on pre-caching. The method first separates question-answer pairs consisting of doctor questions and patient answers from the standard medical consultation dialogue, and uses a multimodal large model to generate digital human materials containing doctor questions and corresponding digital human material storage paths for each question-answer pair, and stores them in a vector database, thereby constructing a pre-cached knowledge base to complete the pre-generation and storage of multimodal interaction materials in high-frequency question-answering scenarios. When a target question input by the user is received, semantic matching is first performed in the vector database. If the match is successful, the target digital human material matching the target question is directly output, skipping the real-time calculation link. If the match is not successful, the original case data of the current patient is used as an external knowledge base, and the multimodal large model is called to generate the text answer corresponding to the target question in real time, and the target question and text answer are combined into a new question-answer pair, and the digital human material is regenerated and stored in the vector database to achieve cache expansion. The above process uses a pre-caching mechanism to store multimodal interaction materials of high-frequency problems in the vector database in advance. During the actual interaction process, the pre-stored materials can be directly hit through semantic matching in the vector database without triggering the real-time full-link calculation of the multimodal large model. Only when encountering a new problem that is not hit is the real-time inference of the large model started and the results added to the cache, which greatly reduces the call frequency of the real-time calculation of the large model and effectively reduces the computing pressure of the system during the real-time interaction process, thereby improving the efficiency of doctor-patient interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 This is a flow chart of a pre-cached AI standardized patient real-time interaction method provided by one embodiment of the present invention;

[0055] Figure 2 is a flowchart of another pre-cached AI standardized patient real-time interaction method provided by one embodiment of the present invention;

[0056] Figure 3 1 is a schematic diagram of a pre-cached AI standardized patient real-time interaction system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, an embodiment of the present invention provides a pre-cached AI standardized patient real-time interaction method, which may include the following steps:

[0059] Step 101: Obtain at least one question-answer pair based on a standard medical consultation dialogue, generate a digital human material corresponding to each question-answer pair using a multimodal large model, and store it in a vector database to build a pre-cached knowledge base. Each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path;

[0060] Step 102: upon receiving the target question input by the user, a semantic matching operation is performed on the target question in the vector database, and a determination is made as to whether the target question matches a preset question in the vector database. If so, step 103 is executed; otherwise, step 104 is executed.

[0061] Step 103: Outputting target digital human material that matches the target question;

[0062] Step 104: Use the original case data of the current patient as an external knowledge base, and use the multimodal large model to generate a text answer corresponding to the target question. The target question is used as the doctor's question for the current question-answer pair, and the text answer is used as the patient's answer for the current question-answer pair. Return to step 101.

[0063] In an embodiment of the present invention, a method for real-time interaction of AI standardized patients based on pre-caching is provided. The method first separates question-answer pairs consisting of doctor questions and patient answers from the standard medical consultation dialogue, and generates digital human materials containing doctor questions and corresponding digital human material storage paths for each question-answer pair with the help of a multimodal large model, and stores them in a vector database, thereby constructing a pre-cached knowledge base to complete the pre-generation and storage of multimodal interaction materials in high-frequency question-answering scenarios. When a target question input by the user is received, a semantic match is first performed in the vector database. If the match is successful, the target digital human material matching the target question is directly output, skipping the real-time calculation link. If the match is not successful, the original case data of the current patient is used as an external knowledge base, and the multimodal large model is called to generate the text answer corresponding to the target question in real time, and the target question and the text answer are combined into a new question-answer pair, and the digital human material is regenerated and stored in the vector database to achieve cache expansion. The above process uses a pre-caching mechanism to store multimodal interaction materials of high-frequency problems in the vector database in advance. During the actual interaction process, the pre-stored materials can be directly hit through semantic matching in the vector database without triggering the real-time full-link calculation of the multimodal large model. Only when encountering a new problem that is not hit is the real-time inference of the large model started and the results added to the cache, which greatly reduces the call frequency of the real-time calculation of the large model and effectively reduces the computing pressure of the system during the real-time interaction process, thereby improving the efficiency of doctor-patient interaction.

[0064] In order to reduce the real-time consumption pressure of the system on computing resources such as GPU, in one embodiment of the present invention, step 101 in the above embodiment may specifically include the following steps:

[0065] Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer;

[0066] Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model;

[0067] Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path;

[0068] The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

[0069] In an embodiment of the present invention, the TTS large model is a text-to-speech system based on deep learning technology. Through large-scale data training and a complex neural network architecture, it achieves highly natural and high-fidelity speech synthesis. It utilizes hundreds of billions of parameters to capture linguistic patterns, processes text-to-speech mapping through discrete coding, autoregressive, or non-autoregressive modeling techniques, and supports zero-shot speech cloning, multilingual synthesis, and emotion control. Therefore, for each question-answer pair, the patient's response can first be processed by the TTS large model to generate an audio file with precise timestamps, ensuring the temporal traceability of speech synthesis. The lip shape synthesis large model is a deep learning-based AI technology that generates high-precision, natural, and smooth lip movement sequences by analyzing text or audio features. It is widely used in fields such as virtual digital humans, film and television animation, and human-computer interaction. Therefore, the audio file can then be input into the lip shape synthesis large model, and a synchronized video file is generated based on the audio timestamp information, ensuring strict temporal correspondence between lip movements and speech content. Then, the data alignment module is used to merge the audio file and the video file according to the timestamp to generate an audio and video file with a timestamp, and its storage path is recorded to achieve accurate alignment of multimodal data. Finally, using the doctor's question as the key and the storage path of the audio and video file as the value, each question and answer pair is converted into a structured digital human material through the multimodal large model and stored in the vector database to build a pre-cached knowledge base. This process establishes a mapping relationship between the doctor's question and the pre-generated audio and video file storage path through multi-stage modal conversion and alignment of the question and answer pairs. In the subsequent real-time interaction scenario, when the target question input by the user matches the preset question, the system can directly retrieve the corresponding audio and video file storage path from the vector database and quickly call the generated digital human material, thereby significantly reducing the amount of calculation in the real-time interaction stage, reducing the real-time consumption pressure of the system on computing resources such as GPU, and improving interaction efficiency and resource utilization.

[0070] To further improve interaction efficiency, in one embodiment of the present invention, performing a semantic matching operation on the target question in the vector database in step 102 in the above embodiment may specifically include the following steps:

[0071] Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively;

[0072] Determine the cosine similarity between the target question vector and the preset question vector using a first formula;

[0073] The first formula is:

[0074] Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

[0075] In an embodiment of the present invention, when processing a user-entered target question, the target question is first converted into a high-dimensional numerical representation of the target question vector. Simultaneously, the preset questions stored in the vector database are converted into corresponding preset question vectors, providing a computable mathematical representation of the text information. Subsequently, the cosine similarity formula is used to calculate the similarity between the two vectors, with larger values ​​indicating closer semantics between the questions. Through this calculation process, the system can efficiently retrieve the preset questions in the vector database that best match the semantics of the target question.

[0076] In order to reduce the real-time computing pressure of the system, in one embodiment of the present invention, determining whether the target problem matches the preset problem in the vector database in step 103 in the above embodiment may specifically include the following steps:

[0077] Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order;

[0078] Determine whether the confidence of the first ranked item exceeds a preset confidence threshold; if so, execute step 103; otherwise, execute step 104.

[0079] In an embodiment of the present invention, first, based on the calculation result of cosine similarity, a corresponding confidence level is generated for each preset question in the vector database, and all preset questions are sorted in descending order from high to low confidence levels to form an ordered list of matching results. Subsequently, the system automatically detects whether the confidence level of the preset question ranked first exceeds a preset confidence threshold (for example, 0.8). If it exceeds the threshold, it indicates that the target question and the preset question are highly semantically similar, and it can be directly determined to be a successful match. The pre-cached digital human material is immediately called without the need to perform one-by-one matching calculations on other preset questions. If it does not exceed the threshold, it is determined to be unmatched, triggering real-time inference of the large model, reducing the amount of calculation during vector database retrieval, and avoiding invalid calculations of low-confidence matching results, significantly improving the decision-making efficiency of the interactive process, and further reducing the pressure of real-time calculations. At the same time, the pre-cached multimodal AI standardized patient real-time interaction optimization method can also solve the following problems: (1) Reduce interaction delay: In most cases, the first token reply can be shortened from 200ms to within 50ms; (2) Improve data alignment consistency: The voice and image synchronization error is less than 100ms; (3) Optimize resource utilization: In most cases, the real-time GPU resource usage can be greatly reduced.

[0080] like Figure 2To more clearly illustrate the technical solutions and advantages of the present invention, a method for real-time interaction with AI standardized patients based on pre-caching is described in detail below. Specifically, the method may include the following steps:

[0081] Step 201: Obtain at least one question-answer pair based on a standard medical consultation dialogue, and use a large TTS model to output the patient's answer contained in each question-answer pair as an audio file with a timestamp, where each question-answer pair includes a doctor's question and a patient's answer;

[0082] For example, a question-answer pair could be: [Doctor: Hello, I feel uncomfortable somewhere. Patient: Doctor, I feel suffocated and chest tightness.]

[0083] Step 202: using the lip shape synthesis model to output the audio file with the timestamp into a video file with the timestamp;

[0084] Step 203: Utilize the data alignment module to merge the audio file and the video file according to the timestamp, generate an audio and video file with a timestamp, and record the storage path;

[0085] Step 204: Using the doctor's question as the key and the storage path of the audio and video file with the timestamp as the value, the multimodal large model is used to generate a digital human material corresponding to each question-answer pair and store it in the vector database to build a pre-cached knowledge base;

[0086] Step 205: upon receiving the target question input by the user, converting the target question and the preset question into a target question vector and a preset question vector respectively;

[0087] Step 206: Determine the cosine similarity between the target question vector and the preset question vector using the first formula;

[0088] Specifically, the first formula is:

[0089] Among them, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

[0090] Step 207: Based on the calculation result of the cosine similarity, generate a confidence score for each preset question in the vector database and sort them in descending order;

[0091] Step 208: Determine whether the confidence of the first ranked item exceeds a preset confidence threshold. If so, execute step 209; otherwise, execute step 210.

[0092] Step 209: Outputting target digital human material that matches the target question;

[0093] Step 210: Use the original case data of the current patient as an external knowledge base, utilize the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question for the current question-answer pair, and the text answer as the patient's answer for the current question-answer pair, and return to execute step 201.

[0094] like Figure 3 As shown, an embodiment of the present invention provides a pre-cached AI standardized patient real-time interaction system, the system comprising:

[0095] Knowledge base construction module 301: is used to obtain at least one question-answer pair based on the standard medical consultation dialogue, generate a digital human material corresponding to each question-answer pair using a multimodal large model, and store it in a vector database to construct a pre-cached knowledge base, wherein each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path;

[0096] Semantic matching module 302: configured to, upon receiving a target question input by a user, perform a semantic matching operation on the target question in the vector database constructed by the knowledge base construction module 301, and determine whether the target question matches a preset question in the vector database; if so, trigger output module 303; otherwise, execute trigger processing module 304;

[0097] The output module 303 is configured to output target digital human materials matching the target question when triggered;

[0098] The processing module 304 is used to, when triggered, use the original case data of the current patient as an external knowledge base, use the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question of the current question and answer pair, and the text answer as the patient's answer to the current question and answer pair, and return to execute the triggering knowledge base construction module 301.

[0099] like Figure 3 As shown, the knowledge base construction module 301 is used to execute:

[0100] Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer;

[0101] Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model;

[0102] Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path;

[0103] The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

[0104] like Figure 3 As shown, the semantic matching module 302 is further configured to execute, upon receiving a target question input by a user:

[0105] Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively;

[0106] Determine the cosine similarity between the target question vector and the preset question vector using a first formula;

[0107] The first formula is:

[0108] Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

[0109] like Figure 3 As shown, the semantic matching module 302 is further configured to, after determining the cosine similarity between the target question vector and the preset question vector using the first formula, execute:

[0110] Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order;

[0111] Determine whether the confidence of the first ranked item exceeds a preset confidence threshold; if so, trigger the output module 303; otherwise, execute the trigger processing module 304.

[0112] It is understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on a pre-cached AI standardized patient real-time interaction system. In other embodiments of the present invention, a pre-cached AI standardized patient real-time interaction system may include more or fewer components than illustrated, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] The information interaction, execution process, etc. between the units in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0114] An embodiment of the present invention further provides a pre-caching-based AI standardized patient real-time interaction system, comprising: at least one memory and at least one processor;

[0115] at least one memory for storing a machine-readable program;

[0116] At least one processor is configured to call a machine-readable program to execute a pre-cached AI standardized patient real-time interaction method according to any embodiment of the present invention.

[0117] An embodiment of the present invention further provides a computer-readable medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the processor executes a pre-cached AI standardized patient real-time interaction method according to any embodiment of the present invention.

[0118] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0119] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0120] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0121] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0122] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0123] Each embodiment of the present invention has at least the following beneficial effects:

[0124] 1. In an embodiment of the present invention, a method for real-time interaction of AI standardized patients based on pre-caching is provided. The method first separates question-answer pairs consisting of doctor questions and patient answers from the standard medical consultation dialogue, and generates digital human materials containing doctor questions and corresponding digital human material storage paths for each question-answer pair with the help of a multimodal large model, and stores them in a vector database, thereby constructing a pre-cached knowledge base to complete the pre-generation and storage of multimodal interaction materials in high-frequency question-answering scenarios. When a target question input by the user is received, a semantic match is first performed in the vector database. If the match is successful, the target digital human material matching the target question is directly output, skipping the real-time calculation link. If the match is not successful, the original case data of the current patient is used as an external knowledge base, and the multimodal large model is called to generate the text answer corresponding to the target question in real time, and the target question and text answer are combined into a new question-answer pair, and the digital human material is regenerated and stored in the vector database to achieve cache expansion. The above process uses a pre-caching mechanism to pre-store multimodal interaction materials for high-frequency issues in the vector database. This allows for direct semantic matching of pre-stored materials during actual interactions, eliminating the need to trigger real-time, full-link computation of the multimodal large model. Real-time inference of the large model is initiated only when a new, unanswered question is encountered, and the results are added to the cache. This significantly reduces the frequency of real-time computation calls to the large model, effectively alleviating the computational pressure on the system during real-time interactions and thereby improving the efficiency of doctor-patient interactions.

[0125] 2. In an embodiment of the present invention, for each question-answer pair, the patient's answer content is first processed through the TTS large model to generate an audio file with a precise timestamp to ensure that the time dimension of the speech synthesis is traceable. The audio file is then input into the lip synthesis large model, and a synchronized video file is generated based on the timestamp information of the audio, so that the lip movements and the speech content strictly correspond in time series. Subsequently, the data alignment module is used to merge the audio file and the video file according to the timestamp to generate an audio and video file with a timestamp, and its storage path is recorded to achieve precise alignment of multimodal data. Finally, using the doctor's question as the key and the storage path of the audio and video file as the value, each question-answer pair is converted into structured digital human material through the multimodal large model, stored in the vector database, and a pre-cached knowledge base is constructed. This process establishes a mapping relationship between the doctor's question and the pre-generated audio and video file storage path by performing multi-stage modal conversion and alignment on the question-answer pair. In subsequent real-time interaction scenarios, when the user's input question matches the pre-stored doctor's question, the system can directly retrieve the corresponding audio and video file storage path from the vector database and quickly call the generated digital human material, thereby significantly reducing the amount of computation in the real-time interaction stage, reducing the system's real-time consumption pressure on computing resources such as GPUs, and improving interaction efficiency and resource utilization.

[0126] 3. In this embodiment of the present invention, when processing a user-entered target question, the target question is first converted into a high-dimensional numerical representation of the target question vector. Simultaneously, the preset questions stored in the vector database are converted into corresponding preset question vectors, providing a computable mathematical representation of the text information. Subsequently, the cosine similarity formula is used to calculate the similarity between the two vectors, with larger values ​​indicating closer semantic alignment of the questions. This calculation process enables the system to efficiently retrieve the preset questions in the vector database that best match the semantics of the target question.

[0127] It should be noted that not all steps and modules in the above processes and system structure diagrams are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0128] In the above embodiments, the hardware unit can be realized by mechanical means or electrical means. For example, a hardware unit can include permanent dedicated circuits or logic (such as special processors, FPGA or ASIC) to complete the corresponding operations. The hardware unit can also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set up by software to complete the corresponding operations. Concrete implementation (mechanical means or dedicated permanent circuits or temporarily set circuits) can be determined based on the consideration on cost and time.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time interaction method for AI standardized patients based on pre-caching, characterized in that: The method includes: A1: Obtain at least one question-answer pair based on a standard medical consultation dialogue, use a multimodal large model to generate a digital human material corresponding to each question-answer pair, and store it in a vector database to build a pre-cached knowledge base. Each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path; A2: upon receiving a target question input by a user, performing a semantic matching operation on the target question in the vector database, and determining whether the target question matches a preset question in the vector database; if so, executing step A3; otherwise, executing step A4; A3: Output the target digital human material that matches the target problem; A4: Use the original case data of the current patient as an external knowledge base, utilize the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question for the current question-answer pair, and the text answer as the patient's answer for the current question-answer pair, and return to execute step A1.

2. The method according to claim 1, characterized in that Said A1 includes: Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer; Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model; Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path; The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

3. The method according to claim 1, characterized in that The step A2 of performing a semantic matching operation on the target question in the vector database includes: Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively; Determine the cosine similarity between the target question vector and the preset question vector using a first formula; The first formula is: Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

4. The method according to claim 3, characterized in that The step A2 of determining whether the target question matches a preset question in the vector database includes: Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order; Determine whether the confidence of the first ranked item exceeds a preset confidence threshold. If so, execute step A3; otherwise, execute step A4.

5. AI standardized patient real-time interaction system based on pre-caching, characterized by: The system includes: A knowledge base construction module is used to obtain at least one question-answer pair based on a standard medical consultation dialogue, generate a digital human material corresponding to each question-answer pair using a multimodal large model, and store it in a vector database to construct a pre-cached knowledge base. Each question-answer pair includes a doctor's question and a patient's answer, and the digital human material includes: the doctor's question and the digital human material storage path; Semantic matching module: configured to, upon receiving a target question input by a user, perform a semantic matching operation on the target question in the vector database and determine whether the target question matches a preset question in the vector database; if so, trigger the output module; otherwise, execute the trigger processing module; The output module is configured to output target digital human materials matching the target question when triggered; The processing module is used to, when triggered, use the original case data of the current patient as an external knowledge base, use the multimodal large model to generate a text answer corresponding to the target question, and use the target question as the doctor's question of the current question and answer pair, and the text answer as the patient's answer to the current question and answer pair, and return to execute the triggering knowledge base construction module.

6. The system according to claim 5, characterized in that The knowledge base construction module is used to perform: Obtaining the at least one question-answer pair split based on the standard medical consultation dialogue, and outputting the patient's answer contained in each question-answer pair as an audio file with a timestamp using a large TTS model, wherein each question-answer pair includes a doctor's question and a patient's answer; Outputting the audio file with the timestamp into a video file with the timestamp using a lip shape synthesis model; Merging the audio file and the video file according to the timestamp using a data alignment module to generate an audio and video file with a timestamp and record a storage path; The doctor's question is used as the key and the storage path of the audio and video file with the timestamp is used as the value. The multimodal large model is used to generate the digital human material corresponding to each question and answer pair and store it in the vector database to build a pre-cached knowledge base.

7. The system according to claim 5, characterized in that The semantic matching module is further configured to, upon receiving a target question input by a user, execute: Converting the target problem and the preset problem into a target problem vector and a preset problem vector respectively; Determine the cosine similarity between the target question vector and the preset question vector using a first formula; The first formula is: Wherein, cosθ is the cosine similarity between the target question vector and the preset question vector, A is the target question vector, and B is the preset question vector.

8. The system according to claim 7, characterized in that The semantic matching module is further configured to, after determining the cosine similarity between the target question vector and the preset question vector using the first formula, execute: Based on the calculation result of the cosine similarity, generating a confidence score for each preset question in the vector database and sorting them in descending order; Determine whether the confidence of the first ranked item exceeds a preset confidence threshold. If so, trigger the output module; otherwise, execute the trigger processing module.

9. AI standardized patient real-time interaction system based on pre-caching, characterized by: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 4.

10. A computer-readable medium, characterized in that The computer-readable medium stores computer instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 4.

Images