Doctor inquiry training method and system

Abstract

The invention is suitable for the field of medical education training, and discloses a doctor inquiry training method and system, and the method comprises the steps: randomly extracting any case, and generating illness state data according to the extracted case; obtaining first behavior data fed back by the trainee based on the illness state data, updating the illness state data according to the first behavior data, obtaining second behavior data fed back by the trainee based on the updated illness state data, judging whether the fed-back second behavior data comprises a diagnosis result and a treatment scheme or not, if not, storing the second behavior data, and if yes, storing the second behavior data; replacing the first behavior data with second behavior data, and repeatedly executing the previous step and the current step; when the second behavior data comprises the diagnosis result and the treatment scheme, a training analysis report is generated according to the first behavior data and all the second behavior data, the method simulates case interaction to train doctors, the training efficiency can be improved, and the labor cost can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of medical education and training, and in particular to a doctor consultation training method and system. Background Art

[0003] As an important means for doctors to diagnose and treat patients clinically, interview is a skill that every pediatrician must master before practicing, and it must be repeatedly practiced and consolidated throughout the medical learning process. At present, it is common practice to use standardized patients for clinical interview skills training. This process first involves the selection of suitable participants. These participants will receive unified and organized training so that they can imitate patients in real clinical environments through various performance techniques, makeup, and the use of props. Training or evaluation will be conducted with these simulated patients through live interaction or remote video. During this process, the teacher needs to be present throughout the process to supervise the performance of the standardized patients and the performance of the trainees or assessees, and provide feedback and evaluation. However, this training model has a high demand for teacher resources, limits the number of people who can participate in the training, reduces the frequency of training, and thus affects the overall efficiency of the training. Therefore, it is necessary to explore ways to improve the efficiency of training.

[0004] In order to facilitate pediatrician training more effectively, most standardized physical child patient models are generally used, which have extremely high requirements for intelligence level, fidelity, and clinical fit. The disadvantage of this solution is that the symptom representation ability is poor and it is difficult to adapt to different disease scenarios; although the terminal-based model can show that the patient has multiple symptoms on the terminal display, the symptom expression is not intuitive, and the interaction method is quite different from the actual clinical operation. Although the virtual patient model realized by VR / AR and other technologies has a high degree of realism and flexible scalability, the high cost and the need for additional learning of equipment operation methods still hinder its popularization in primary medical institutions.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0006] The first object of the present invention is to provide a doctor consultation training method, which is used to collect and analyze the behavioral data of the trainer during the consultation process through simulated case interaction to train doctors, thereby improving training efficiency and reducing labor costs.

[0007] To achieve the above-mentioned purpose, the solution provided by the present invention is: a doctor consultation training method, the doctor consultation training method comprising: randomly selecting any case and generating condition data according to the selected case; obtaining first behavior data fed back by the trainer based on the condition data, the first behavior data including first consultation audio data or including first consultation audio data and first action video data of examining a simulated patient; updating the condition data according to the first behavior data, and obtaining second behavior data fed back by the trainer based on the updated condition data, the second behavior data at least including any one of second consultation audio data and second action video data of examining a simulated patient; judging whether the fed back second behavior data includes a diagnosis result and a treatment plan, if not, saving the second behavior data, replacing the first behavior data with the second behavior data, and returning to execute updating the condition data according to the first behavior data, and obtaining second behavior data fed back by the trainer based on the updated condition data, the second behavior data at least including any one of second consultation audio data and second action video data of examining a simulated patient; when the second behavior data includes a diagnosis result and a treatment plan, generating a training analysis report according to the first behavior data and all the second behavior data.

[0008] Preferably, the randomly extracting any one case and generating condition data based on the extracted case includes: randomly extracting any one case from a pre-built case database; inputting the extracted case into a random operator generator to generate virtual patient medical test data; and generating condition data based on the patient medical test data.

[0009] Preferably, the randomly extracting any case from a pre-constructed case database includes: collecting medical record data of real patients; desensitizing the collected medical record data to obtain desensitized medical record data; constructing a case database based on the desensitized medical record data, and randomly extracting any case from the pre-constructed case database.

[0010] Preferably, the collected medical record data is desensitized, and before obtaining the desensitized medical record data, the collected medical record data is also cleaned.

[0011] Preferably, generating the condition data based on the patient's medical examination data includes: if the condition data is a text description, inputting the patient's medical examination data into a large medical model to generate a text description of the condition, and using the text description of the condition as the condition data; if the condition data is voice data, inputting the patient's medical examination data into a large medical model to generate a text description of the condition, converting the text description of the condition into voice data, and using the voice data as the condition data; if the condition data includes a text description and voice data, inputting the patient's medical examination data into a large medical model to generate a text description of the condition, converting the text description of the condition into voice data, and using the text description of the condition and the voice data as the condition data.

[0012] Preferably, when the second behavior data includes a diagnosis result and a treatment plan, a training analysis report is generated based on the first behavior data and all the second behavior data, including: if the first behavior data includes first consultation audio data, and the second behavior data includes second action video data, a diagnosis result and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second action video data, the diagnosis result and the treatment plan; if the first behavior data includes first consultation audio data, and the second behavior data includes second consultation audio data, a diagnosis result and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the diagnosis result and the treatment plan; if the first behavior data includes first consultation audio data, and the second behavior data includes second consultation audio data, second action video data, a diagnosis result and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the second action video data, the diagnosis result and the treatment plan; The first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, diagnosis results and treatment plans, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the first action video data, the diagnosis results and the treatment plans; if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second action video data, diagnosis results and treatment plans, then a training analysis report is generated based on the first consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plans; if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plans.

[0013] Preferably, if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plans, specifically including: if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then the trainer's consultation skills are evaluated based on the first consultation audio data and the second consultation audio data to obtain a consultation score; the trainer's examination behavior is evaluated based on the first action video data and the second action video data to obtain an examination score; the trainer's diagnostic accuracy is evaluated based on the diagnosis results and the treatment plans to obtain a diagnosis score; and a training analysis report is generated based on the consultation score, the examination score and the diagnosis score.

[0014] Preferably, evaluating the trainer's inspection behavior according to the first action video data and the second action video data to obtain an inspection score includes: splitting the first action video data and the second action video data into multiple detection images; using the MobilePose model to process the multiple detection images to obtain the two-dimensional coordinate data of the skeleton key points in each detection image; and using a multi-layer perceptron model to evaluate the trainer's inspection behavior according to the two-dimensional coordinate data of the skeleton key points to obtain an inspection score.

[0015] The second object of the present invention is to provide a doctor consultation training system, which is used to execute the doctor consultation training method as described above, and the doctor consultation training system includes a user terminal and a cloud server communicating with the user terminal, the user terminal includes a simulated patient entity model, a display unit, a speaker unit, and a data acquisition unit, the display unit is used to display the condition data, the speaker unit is used to play the voice data in the condition data, the data acquisition unit is used for the first behavior data and the second behavior data, the cloud server includes a data processing unit, a data generation unit, a training evaluation unit and a storage unit, the data processing unit is connected to the data acquisition unit and the data generation unit respectively, and the data processing unit is used to parse the first behavior The data and the second behavior data are used to obtain the trainer's inquiry content, the data generating unit is connected to the display unit and the speaker unit respectively, the data generating unit is used to randomly extract any case, generate the condition data according to the extracted case, and send the condition data to the display unit and the speaker unit, the data generating unit is also used to update the condition data according to the query content obtained by the analysis, and send the updated condition data to the display unit and the speaker unit, the training evaluation unit is connected to the data acquisition unit, the training evaluation unit is used to obtain the diagnosis results and treatment plans given by the trainer, and generate a training analysis report according to the first behavior data and all the second behavior data, and the storage unit is used to store the collected data and the training analysis report.

[0016] In the solution provided by the present invention, by randomly extracting cases and generating disease data, the consultation process in a real clinical environment can be highly simulated. Trainers are faced with diverse cases and dynamically changing disease data, which helps trainers improve their adaptability and decision-making capabilities in complex clinical situations. During the consultation process, voice dialogue interaction and medical behavior recognition functions allow trainers to integrate into the simulation scene with higher authenticity. Moreover, the disease data will be updated in real time according to the trainer's inquiry content. This dynamic interaction mechanism provides trainers with instant feedback, and trainers can adjust the consultation strategy according to changes in the disease. This real-time interaction and feedback loop helps to deepen their understanding of the question. In addition, in addition to the audio data of the consultation, the second action video data of the trainer examining the simulated patient is also collected. This multimodal data collection method can more comprehensively reflect the trainer's clinical skills and behavioral performance. By analyzing these data, the trainer's communication skills, observation, judgment and practical operation ability can be evaluated, providing a more objective and comprehensive evaluation of the training effect. At the same time, a training analysis report is generated based on the collected data, diagnosis results and treatment plan. The evaluation results of the trainer's entire consultation process replace the evaluation of the teacher with the deep learning method, which can improve training efficiency and reduce labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0018] Figure 1 is a flow chart of a doctor consultation training method provided by an embodiment of the present invention;

[0019] Figure 2 It is a structural block diagram of a doctor consultation training system provided by an embodiment of the present invention.

[0020] Description of reference numerals:

[0021] 10. User terminal; 11. Simulated patient entity model; 12. Display unit; 13. Speaker unit; 14. Data acquisition unit; 20. Cloud server; 21. Data processing unit; 22. Data generation unit; 23. Training and evaluation unit; 24. Storage unit. DETAILED DESCRIPTION

[0022] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 In an embodiment of the present invention, a doctor consultation training method includes:

[0024] S101. Randomly select any case and generate disease data according to the selected case;

[0025] S102, obtaining first behavior data based on the condition data feedback of the trainer, wherein the first behavior data includes first consultation audio data or includes the first consultation audio data and first action video data of examining a simulated patient;

[0026] S103, updating the condition data according to the first behavior data, and obtaining second behavior data fed back by the trainer based on the updated condition data, wherein the second behavior data includes at least any one of second consultation audio data and second action video data of examining a simulated patient;

[0027] S104, determining whether the fed-back second behavior data includes a diagnosis result and a treatment plan, if not, saving the second behavior data, replacing the first behavior data with the second behavior data, returning to execute updating the condition data according to the first behavior data, and obtaining the second behavior data fed back by the trainer based on the updated condition data, wherein the second behavior data includes at least any one of the second consultation audio data and the second action video data of the examination simulated patient;

[0028] S105: When the second behavior data includes a diagnosis result and a treatment plan, generating a training analysis report according to the first behavior data and all the second behavior data.

[0029] It can be understood that the doctor consultation training method of this embodiment is suitable for pediatrician training, and the randomly selected cases are children's cases, especially cases of common children's diseases.

[0030] Common childhood diseases are the most common conditions encountered in pediatric medical practice, including respiratory infections (such as colds and flu), diarrhea, skin problems, etc. The high incidence and prevalence of these diseases means that medical staff need to be proficient in their diagnosis, treatment, and prevention methods. By training on these common cases, on the one hand, it can ensure that medical staff can respond quickly and accurately when faced with these common conditions. On the other hand, being familiar with the diagnostic process and treatment methods of common cases can make medical staff more efficient in clinical practice, help reduce the risk of misdiagnosis and missed diagnosis, shorten patients' waiting time and treatment cycle, and improve the overall efficiency of medical services.

[0031] It can be understood that the first behavior data may include only the first consultation audio data, or may include both the first consultation audio data and the first action video data of examining a simulated patient.

[0032] Similarly, the second behavior data may include only the second consultation audio data, or only the second action video data of examining a simulated patient, or may include both the second consultation audio data and the second action video data of examining a simulated patient.

[0033] It is understandable that during the training process, the trainer may ask multiple questions and detect the patient's behavioral actions multiple times. Therefore, it is necessary to perform steps 103 and 104 multiple times. When collecting data, multiple copies of the second consultation audio data and multiple copies of the second action video data may be collected.

[0034] In this embodiment, the condition data may be a text description of the condition, may be voice data, or may include both a text description of the condition and voice data.

[0035] In this embodiment, by randomly selecting cases and generating disease data, the consultation process in a real clinical environment can be highly simulated. Trainers are faced with diverse cases and dynamically changing disease data, which helps trainers improve their adaptability and decision-making capabilities in complex clinical situations. During the consultation process, voice dialogue interaction and medical behavior recognition functions allow trainers to integrate into the simulation scene with higher authenticity. Moreover, the disease data will be updated in real time according to the trainer's inquiry content. This dynamic interaction mechanism provides trainers with instant feedback, and trainers can adjust the consultation strategy according to changes in the disease. This real-time interaction and feedback loop helps to deepen their understanding of the consultation process. In addition, in addition to the audio data of the consultation, the second action video data of the trainer examining the simulated patient is also collected. This multimodal data collection method can more comprehensively reflect the trainer's clinical skills and behavioral performance. By analyzing these data, the trainer's communication skills, observation, judgment and practical operation ability can be evaluated, providing a more objective and comprehensive evaluation of the training effect. At the same time, a training analysis report is generated based on the collected data, diagnosis results and treatment plan. The evaluation results of the trainer's entire consultation process replace the evaluation of the teaching evaluator with the deep learning method, which can improve training efficiency and reduce labor costs.

[0036] In this embodiment, in step S101, any case is randomly selected, and condition data is generated based on the selected case, including: randomly selecting any case from a pre-built case database; inputting the selected case into a random operator generator to generate virtual patient medical examination data; and generating condition data based on the patient medical examination data.

[0037] In this embodiment, any case is randomly selected from a pre-constructed case database, including: collecting medical record data of real patients; desensitizing the collected medical record data to obtain desensitized medical record data; constructing a case database based on the desensitized medical record data, and randomly selecting any case from the pre-constructed case database.

[0038] In this embodiment, the medical record data includes clinical records, examination reports and diagnosis information.

[0039] In this embodiment, the collected medical record data is desensitized. Before obtaining the desensitized medical record data, the collected medical record data is also cleaned. The data cleaning of the collected medical record data specifically includes:

[0040] Data deduplication: Remove duplicate records to ensure the uniqueness of each record in the database.

[0041] Data format unification: unify the data formats from different sources, such as date format, coding standards, etc.

[0042] Outlier processing: Identify and process outliers in data, such as unreasonable values, missing values, etc.

[0043] In this embodiment, the case database contains basic information of each case, common symptoms, range of medical examination indicators, etc. The case database can be a relational database (such as MySQL, PostgreSQL) or a non-relational database (such as MongoDB).

[0044] In this embodiment, the random operator generator is intended to generate virtual patient medical examination data by applying a series of randomization processes and medical knowledge rules. These data include but are not limited to blood routine, biochemical indicators, imaging examination results, etc., which are used to simulate the health status and disease status of real patients.

[0045] Optionally, generating condition data based on the patient's medical test data specifically includes: if the condition data is a text description, inputting the patient's medical test data into the medical big model to generate a text description of the condition, and using the text description of the condition as the condition data; if the condition data is voice data, inputting the patient's medical test data into the medical big model to generate a text description of the condition, converting the text description of the condition into voice data, and using the voice data as the condition data; if the condition data includes text description and voice data, inputting the patient's medical test data into the medical big model to generate a text description of the condition, converting the text description of the condition into voice data, and using the text description of the condition and the voice data as the condition data.

[0046] In this embodiment, the medical big model is a trained natural language processing (NLP) model, such as BERT, GPT, etc., to generate a text description of the condition. The natural language processing (NLP) model is pre-trained on a large amount of medical text data to understand medical terms and condition descriptions.

[0047] In this embodiment, cases are randomly extracted from the case database to ensure the diversity and authenticity of the simulation, and a random operator generator is used to automatically generate virtual patient medical test data, and the disease data is further generated based on these data, which greatly improves efficiency. In addition, according to needs, the disease data can be flexibly presented in the form of text description, voice data, or a combination of the two, which enriches the expression of the disease data and increases its authenticity.

[0048] In this embodiment, in step S102, after the medical condition data is sent to the trainer, the trainer may interview the patient based on the medical condition data. Therefore, the voice data of the trainer interviewing the patient based on the medical condition data can be collected as the first consultation audio data. The trainer may also interview the patient based on the medical condition data and examine the simulated patient. Therefore, the voice data of the trainer interviewing the patient based on the medical condition data can be collected as the first consultation audio data, and the image data of the trainer examining the patient can be collected through the camera as the first action video data.

[0049] In this embodiment, in step S103, the first consultation data is parsed to obtain the trainer's inquiry content, and the condition data is updated according to the inquiry content, and the updated condition data is sent to the trainer. The trainer continues to diagnose the patient according to the updated condition data. During the consultation process, a consultation dialogue may occur, and the simulated patient may continue to be examined. This process may occur multiple times until the trainer completes the consultation and gives a diagnosis result and treatment plan. Therefore, the voice data of the trainer's consultation with the patient according to the updated condition data can be collected as the second consultation audio data, and the image data of the trainer's examination of the patient can be collected by the camera as the second action video data.

[0050] In this embodiment, in step S104, when the second behavior data includes a diagnosis result and a treatment plan, a training analysis report is generated based on the first behavior data and all the second behavior data, including: if the first behavior data includes first consultation audio data, and the second behavior data includes second action video data, a diagnosis result, and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second action video data, the diagnosis result, and the treatment plan; if the first behavior data includes first consultation audio data, and the second behavior data includes second consultation audio data, a diagnosis result, and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the diagnosis result, and the treatment plan; if the first behavior data includes first consultation audio data, and the second behavior data includes second consultation audio data, second action video data, a diagnosis result, and a treatment plan, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the diagnosis result, and the treatment plan Report; if the first behavior data includes the first consultation audio data and the first action video data, and the second behavior data includes the second consultation audio data, the diagnosis result and the treatment plan, then a training analysis report is generated according to the first consultation audio data, the second consultation audio data, the first action video data, the diagnosis result and the treatment plan; if the first behavior data includes the first consultation audio data and the first action video data, and the second behavior data includes the second action video data, the diagnosis result and the treatment plan, then a training analysis report is generated according to the first consultation audio data, the first action video data, the second action video data, the diagnosis result and the treatment plan; if the first behavior data includes the first consultation audio data and the first action video data, and the second behavior data includes the second consultation audio data, the second action video data, the diagnosis result and the treatment plan, then a training analysis report is generated according to the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis result and the treatment plan.

[0051] In this embodiment, the training analysis report includes the consultation score, the examination score, the diagnosis score, and the deficiencies in the consultation process.

[0052] Furthermore, the training analysis report can also score the trainer's emotions.

[0053] Optionally, if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plans, specifically including: if the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then the trainer's consultation skills are evaluated based on the first consultation audio data and the second consultation audio data to obtain a consultation score; the trainer's examination behavior is evaluated based on the first action video data and the second action video data to obtain an examination score; the trainer's diagnostic accuracy is evaluated based on the diagnosis results and treatment plans to obtain a diagnosis score; and a training analysis report is generated based on the consultation score, examination score and diagnosis score.

[0054] Similarly, if only the first consultation audio data exists, the trainer's consultation skills are evaluated based on the first consultation audio data to obtain a consultation score; if only the first action video data exists, the trainer's inspection behavior is evaluated based on the first action video data and the second action video data to obtain an inspection score.

[0055] In this embodiment, speech recognition technology is used to convert the audio data of the first consultation audio data and the second consultation audio data into text data, and then the text data is analyzed to evaluate the trainer's consultation skills. Finally, the consultation score is calculated based on the evaluation results of the consultation skills.

[0056] In this embodiment, the trainer's questioning skills are evaluated, that is, whether the trainer follows the correct questioning process, whether the trainer uses appropriate questioning methods and techniques, and whether the necessary patient information is successfully obtained. Among them, asking about the disease condition (such as symptoms, course of disease, past medical history, etc.) is a key link in the questioning process, and therefore is also one of the important indicators for evaluating questioning skills.

[0057] In this embodiment, the trainee's inspection score is evaluated according to the first action video data and the second action video data, including: splitting the first action video data and the second action video data into multiple detection images; using the MobilePose model to process the multiple detection images to obtain the two-dimensional coordinate data of the skeleton key points in each detection image; using the multi-layer perceptron model to evaluate the trainee's inspection behavior according to the two-dimensional coordinate data of the skeleton key points to obtain the inspection score.

[0058] MobilePose is a bottom-up posture detection method, which means that the local key points in the image (such as the left wrist, right knee, etc.) are first detected, and then these local information are combined to infer the overall posture. MobilePose consists of two parts: the feature network and the prediction network. The main steps include: preprocessing the detected image, including resizing, normalization, etc.; using the processed image as the input of the feature network, and outputting a feature map based on the MobileNetV2 feature network; inputting the feature map into the prediction network, that is, a multi-stage two-branch network based on MobileNetV2, the upper branch is used to predict the part affinity domain, characterizing the position and direction information between the key points; the lower branch is used to predict the part position confidence map, characterizing the position of the key points.

[0059] It can be understood that the part affinity field matches each key point with a person. It indicates whether the connection between two key points of a person (i.e. limbs) exists at a certain pixel point. If so, its value is the unit vector of the connection, otherwise it is 0. Therefore, the part affinity field is used to establish the position information and direction information of the connection between key points. The confidence map indicates the confidence that a key point appears at a certain pixel position. Since the algorithm uses a total of 18 human key points, the network will generate 18 corresponding confidence maps.

[0060] It can be understood that the characteristic of MobileNetV2 is its inverted residual structure, that is, it first increases the dimension and then reduces the dimension. The design of the inverted residual block can save the computational cost of the network architecture, and the dilated convolution in the inverted residual block increases the information interaction between channels, so that the network can better capture features.

[0061] In this embodiment, during the inspection phase, the trainee's movements can be divided into 12 basic movements, including hand-raising, foot-raising, shirt-lifting to see the back, neck and abdomen pressing, tongue pressing, heart rate measurement, and heart sound listening. In order to improve the classification effect and reduce redundant input, the system only retains the position changes of 8 key points of the left and right hands, including the shoulder, elbow, wrist, nose, and neck, representing the overall displacement of the head and hands, and discards the key point information of the legs and head, so that the multi-layer perceptron model focuses on learning physical examination operations based on hand movements.

[0062] The multi-layer perceptron model is a feed-forward artificial neural network model that can perform nonlinear transformations on input data to achieve complex pattern recognition and classification tasks. In the field of human behavior recognition, the multi-layer perceptron model can receive 2D coordinate data of key points of the human body as input, and identify the trainer's inspection behavior by learning the change patterns of these key points over time, and give corresponding scores.

[0063] In this embodiment, the multi-layer perceptron model includes an input layer, a first hidden layer, a second hidden layer, and an output layer. Nonlinearity is introduced between two adjacent layers through the ReLU activation function. The input layer receives the two-dimensional coordinate data of the skeleton key points. The first hidden layer includes a plurality of neurons for preliminarily extracting and converting the features of the input data, and the second hidden layer further extracts and combines the features from the first hidden layer. The output layer includes one or more neurons, and the output layer includes a neuron that receives the weighted input from the second hidden layer and outputs the inspection score through a linear activation function.

[0064] In this embodiment, the two-dimensional coordinate data of the skeleton key points may be the key point coordinates in a single frame image, or may be a changing sequence of the key point coordinates in consecutive frames.

[0065] Similarly, if there is only the second action video data but no second consultation audio data, the trainee's consultation skills are evaluated based on the first consultation audio data and the second consultation audio data to obtain a consultation score, and the rest is the same as above.

[0066] Similarly, if there is only the second consultation audio data but no second action video data, the trainer's inspection behavior is evaluated based on the first action video data to obtain an inspection score, and the rest is the same as described above.

[0067] The present invention also provides a doctor consultation training system, including a user terminal 10 and a cloud server 20 communicating with the user terminal 10, the user terminal 10 includes a simulated patient entity model 11, a display unit 12, a speaker unit 13, and a data acquisition unit 14, the display unit 12 is used to display the condition data, the speaker unit 13 is used to play the voice data in the condition data, the data acquisition unit 14 is used for the first behavior data and the second behavior data, the cloud server 20 includes a data processing unit 21, a data generation unit 22, a training evaluation unit 23 and a storage unit 24, the data processing unit 21 is respectively connected to the data acquisition unit 14 and the data generation unit 22, the data processing unit 21 is used to parse the first behavior data and the second behavior data to obtain the inquiry content of the trainer, and the data generation unit 22 is respectively Connected to the display unit 12 and the speaker unit 13, the data generation unit 22 is used to randomly extract any case, generate disease data according to the extracted case, and send the disease data to the display unit 12 and the speaker unit 13. The data generation unit 22 is also used to update the disease data according to the query content obtained by analysis, and send the updated disease data to the display unit 12 and the speaker unit 13. The training evaluation unit 23 is connected to the data acquisition unit 14. The training evaluation unit 23 is used to obtain the diagnosis results and treatment plans given by the trainer, and generate a training analysis report based on the collected data, diagnosis results and treatment plans. The storage unit 24 is used to store the collected data and the training analysis report. The trainer can use the user terminal 10 to access the cloud server 20 to understand his own training evaluation conclusions.

[0068] In this embodiment, illustratively, the simulated patient entity model 11 is a child simulated patient, and the speaker unit 13 is used to play the disease condition data and, at the same time, can also be used to play the crying sound of a child.

[0069] In this embodiment, the data acquisition unit 14 includes a voice acquisition device and an image acquisition device, which is used to collect video information such as the trainer's voice conversation and physical examination operations during the training process, namely, the first consultation audio data, the first action video data of examining the simulated patient, the second consultation audio data and the second action video data of examining the simulated patient, the diagnosis results and the treatment plan.

[0070] It can be understood that one cloud service can communicate with multiple user terminals 10 to cooperate with multiple user terminals 10 for training.

[0071] In this embodiment, a virtual child with hand, foot and mouth disease is used as an example to explain the workflow of the pediatrician consultation training system. The "trainer" in the following text is the "user" in the above text.

[0072] First, the display unit 12 displays the medical data of the virtual child: the child is male, 2 years old, has a cough, and has a rash on the hands and mouth.

[0073] Subsequently, the trainer asks a question based on the medical condition data: "Excuse me, are you feeling unwell?", forming the first medical consultation audio data. The first medical consultation audio data is collected by the data collection unit 14 and transmitted to the data processing unit 21. The data processing unit 21 analyzes the first medical consultation audio data to obtain the trainer's inquiry content, and then the data generation unit 22 generates updated medical condition data, for example, generating a virtual parent voice answer "He has been coughing for 3 days, and he has convulsions from time to time. There are rashes on his hands and mouth. He feels a little feverish and is not in good spirits."

[0074] During the medical consultation process, if the trainee performs a simulated physical examination on the physical child model, the specific operations are: 1) raising hands; 2) raising feet; 3) lifting shirt to see the back; 4) pressing the neck and abdomen; 5) taking tongue depressor and pressing tongue; 6) taking stethoscope to measure heart rate and listen to heart sounds, the data acquisition unit 14 collects the trainee's movements of examining the simulated patient, forms the first movement video data, and transmits it to the training evaluation unit 23.

[0075] Next, the trainer asks questions based on the updated medical condition data: "Does he attend childcare? Has he come into contact with other sick children? Does he have any drug allergies?", forming the second medical consultation audio data. The second medical consultation audio data is collected by the data collection unit 14 and transmitted to the data processing unit 21. The data processing unit 21 analyzes the second medical consultation audio data to obtain the trainer's inquiry content, and then the data generation unit 22 generates updated medical condition data, for example, generating a virtual parent answer "There are other children in the childcare center who have similar symptoms and are sick and have asked for leave. He is not allergic to drugs." At the same time, the trainer points out in voice what other examinations the child needs to do: blood routine, blood electrolytes, blood sugar, chest X-ray.

[0076] During the medical consultation process, if the trainee performs a simulated physical examination on the physical child model, the specific operations are: 1) raising hands; 2) raising feet; 3) lifting shirt to see the back; 4) pressing the neck and abdomen; 5) taking tongue depressor and pressing tongue; 6) taking stethoscope to measure heart rate and listen to heart sounds, the data acquisition unit 14 collects the trainee's movements of examining the simulated patient, forms the second movement video data, and transmits it to the training evaluation unit 23.

[0077] Next, it is determined whether the second behavior data includes the diagnosis result and treatment plan. Assuming that the second behavior data does not include the diagnosis result and treatment plan, the second behavior data is saved in the storage unit 24, and the data generation unit 22 updates the condition data according to the saved second behavior data. For example, the data generation unit 22 generates the blood routine, blood electrolyte, blood sugar, chest X-ray and other examination results of the virtual child, and displays them to the trainer through the display unit 12. The trainer analyzes the examination results and gives the diagnosis result and treatment plan. For example, the trainer answers in voice, "After analyzing the above situation, the child is clinically diagnosed with severe hand, foot and mouth disease. Because the child has an acute / fast onset, he is a preschooler, with fever and rash on the hands, feet, mouth, and buttocks, and the child has nervous system / respiratory system involvement. The child needs to be hospitalized as soon as possible, and further examination and treatment are carried out." The data acquisition unit 14 collects new second behavior data, that is, the above-mentioned diagnosis result and treatment plan.

[0078] Continue to determine whether the latest second behavior data includes diagnosis results and treatment plans. Since the latest second behavior data includes diagnosis results and treatment plans, the training evaluation unit 23 generates a training analysis report based on the first consultation audio data, the first action video data, the second consultation audio data, the second action video data, the diagnosis results and the treatment plan during the consultation process, and stores it in the storage unit 24. The trainer can see the evaluation results and error records of the entire consultation process on the display unit 12, and this pediatric consultation training is completed.

[0079] It is understandable that if the trainer needs multiple consultations to be able to give a diagnosis result and treatment plan, that is, there are multiple second consultation audio data and multiple second action video data, then when generating a training analysis report, it is necessary to combine the first consultation audio data, the first action video data, multiple second consultation audio data, multiple second action video data, the diagnosis result and treatment plan for analysis and processing.

[0080] In this embodiment, the data processing unit 21 and the storage unit 24 are placed on the cloud server 20. On the one hand, the user terminal 10 can be simplified. On the other hand, the data processing unit 21 and the storage unit 24 set in the cloud server 20 can be used to simultaneously process and store the working status data of multiple user terminals 10. At the same time, the processing efficiency is high, and there is no need to separately configure an advanced data processing unit 21 and storage for each user terminal 10, which is conducive to improving efficiency, saving costs, and improving reliability.

[0081] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A doctor consultation training method, characterized in that: The doctor consultation training method comprises: Randomly select any case and generate disease data based on the selected case; Acquire first behavior data based on the condition data feedback of the trainer, wherein the first behavior data includes first consultation audio data or includes the first consultation audio data and first action video data of examining a simulated patient; updating the condition data according to the first behavior data, and obtaining second behavior data fed back by the trainer based on the updated condition data, wherein the second behavior data includes at least one of second consultation audio data and second action video data of examining a simulated patient; Determine whether the fed-back second behavior data includes a diagnosis result and a treatment plan, if not, save the second behavior data, replace the first behavior data with the second behavior data, return to execute updating the condition data according to the first behavior data, and obtain the second behavior data fed back by the trainer based on the updated condition data, wherein the second behavior data includes at least any one of the second consultation audio data and the second action video data of the examination simulated patient; When the second behavior data includes a diagnosis result and a treatment plan, a training analysis report is generated according to the first behavior data and all the second behavior data.

2. The doctor consultation training method according to claim 1, characterized in that: The randomly selected case and the generated condition data according to the selected case include: Randomly select any case from the pre-built case database; Inputting the extracted cases into a random operator generator to generate virtual patient medical test data; The condition data is generated according to the patient's medical examination data.

3. The doctor consultation training method according to claim 2, characterized in that: The randomly selected case from the pre-built case database includes: Collect medical records data from real patients; Desensitizing the collected medical record data to obtain desensitized medical record data; A case database is constructed based on the desensitized medical record data, and any case is randomly selected from the pre-constructed case database.

4. The doctor consultation training method according to claim 3, characterized in that: The collected medical record data is desensitized, and before obtaining the desensitized medical record data, the collected medical record data is also cleaned.

5. The doctor consultation training method according to claim 2, characterized in that: The generating of condition data according to the patient's medical examination data comprises: If the condition data is a text description, the patient's medical examination data is input into the medical big model to generate a text description of the condition, and the text description of the condition is used as the condition data; If the condition data is voice data, the patient's medical test data is input into the medical big model to generate a text description of the condition, and the text description of the condition is converted into voice data, and the voice data is used as the condition data; If the condition data includes text description and voice data, the patient medical examination data is input into the medical big model to generate a text description of the condition, and the text description of the condition is converted into voice data, and the text description of the condition and the voice data are used as the condition data.

6. The doctor consultation training method according to claim 1, characterized in that: When the second behavior data includes a diagnosis result and a treatment plan, generating a training analysis report according to the first behavior data and all the second behavior data includes: If the first behavior data includes first medical consultation audio data, and the second behavior data includes second behavior video data, a diagnosis result, and a treatment plan, then generating a training analysis report according to the first medical consultation audio data, the second behavior video data, the diagnosis result, and the treatment plan; If the first behavior data includes first medical consultation audio data, and the second behavior data includes second medical consultation audio data, a diagnosis result, and a treatment plan, then generating a training analysis report according to the first medical consultation audio data, the second medical consultation audio data, the diagnosis result, and the treatment plan; If the first behavior data includes first medical consultation audio data, and the second behavior data includes second medical consultation audio data, second action video data, diagnosis results, and treatment plans, then a training analysis report is generated according to the first medical consultation audio data, the second medical consultation audio data, the second action video data, the diagnosis results, and the treatment plans; If the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, diagnosis results, and treatment plans, then a training analysis report is generated according to the first consultation audio data, the second consultation audio data, the first action video data, the diagnosis results, and the treatment plans; If the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second action video data, a diagnosis result, and a treatment plan, then a training analysis report is generated according to the first consultation audio data, the first action video data, the second action video data, the diagnosis result, and the treatment plan; If the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plan, then a training analysis report is generated based on the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plan.

7. The doctor consultation training method according to claim 6, characterized in that: If the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then a training analysis report is generated according to the first consultation audio data, the second consultation audio data, the first action video data, the second action video data, the diagnosis results and the treatment plans, specifically including: If the first behavior data includes first consultation audio data and first action video data, and the second behavior data includes second consultation audio data, second action video data, diagnosis results and treatment plans, then the consultation skills of the trainee are evaluated according to the first consultation audio data and the second consultation audio data to obtain a consultation score; Evaluate the trainer's inspection behavior according to the first action video data and the second action video data to obtain an inspection score; According to the diagnosis result and the treatment plan, the trainer's diagnostic accuracy is evaluated to obtain a diagnosis score; A training analysis report is generated according to the inquiry score, the examination score and the diagnosis score.

8. The doctor consultation training method according to claim 7, characterized in that: The step of evaluating the trainer's inspection behavior according to the first action video data and the second action video data to obtain an inspection score includes: Splitting the first action video data and the second action video data into a plurality of detection images; Use the MobilePose model to process multiple detection images and obtain the two-dimensional coordinate data of the skeleton key points in each detection image; A multi-layer perceptron model is used to evaluate the trainer's inspection behavior based on the two-dimensional coordinate data of the skeleton key points to obtain the inspection score.

9. A doctor consultation training system, characterized in that: The doctor consultation training system is used to execute the doctor consultation training method according to any one of claims 1 to 8, and the doctor consultation training system includes a user terminal and a cloud server communicating with the user terminal, the user terminal includes a simulated patient entity model, a display unit, a speaker unit, and a data acquisition unit, the display unit is used to display the condition data, the speaker unit is used to play the voice data in the condition data, the data acquisition unit is used for the first behavior data and the second behavior data, the cloud server includes a data processing unit, a data generation unit, a training evaluation unit and a storage unit, the data processing unit is connected to the data acquisition unit and the data generation unit respectively, and the data processing unit is used to parse the first behavior data and the second behavior data The data generating unit is connected to the display unit and the speaker unit respectively, and is used to randomly extract any case, generate condition data according to the extracted case, and send the condition data to the display unit and the speaker unit. The data generating unit is also used to update the condition data according to the query content obtained by analysis, and send the updated condition data to the display unit and the speaker unit. The training evaluation unit is connected to the data acquisition unit, and is used to obtain the diagnosis results and treatment plans given by the trainer, and generate a training analysis report according to the first behavior data and all the second behavior data. The storage unit is used to store the collected data and the training analysis report.

Images