AI training system based on clinical scenario simulation

Abstract

The application discloses an AI training system based on clinical scenario simulation and relates to the field of simulation learning, and comprises a data construction module, a scenario modeling module and a user interface, wherein the data construction module is used for obtaining performance parameters and result parameters of historical clinical cases from a clinical database according to preset clinical operation indexes to form an original training data set; the scenario modeling module is used for providing the user interface, constructing a three-dimensional training model provided with multiple different clinical scenarios based on the original training data set, and generating and projecting guiding video or audio content corresponding to a selected scenario; the application generates diversified training scenes, realizes personalized training, greatly improves the sense of reality of training and the participation of students, collects motion and voice data of students in real time, and immediately converts the data into quantifiable qualified coefficients and specific deviation analysis, greatly reduces the burden of instructors, and accelerates the formation of correct skill patterns.

Description

Technical Field

[0001] This invention relates to the field of simulation learning technology, specifically to an AI training system based on clinical scenario simulation. Background Technology

[0002] The maturity of virtual reality, augmented reality, and high-precision sensor technologies has made it possible to construct highly realistic 3D training environments. The healthcare industry has placed higher demands on standardized procedures, patient safety, and reduced clinical errors, requiring training methods that go beyond traditional practice. Breakthroughs in artificial intelligence, particularly in computer vision, natural language processing, and reinforcement learning, provide the technological foundation for real-time analysis of complex human behavior and the provision of intelligent feedback.

[0003] Traditional and existing clinical simulation training methods rely on instructors' visual observation and post-event recall for assessment, resulting in feedback that is often untimely and inaccurate, making it difficult to pinpoint subtle errors. High-end simulation equipment is expensive, and physical mannequins cannot flexibly generate a large number of variant cases, making it difficult to cover various rare, critical, or sudden complication scenarios, thus limiting the breadth and depth of training. Most existing systems provide fixed scripts, making it impossible to dynamically adjust the difficulty of the scenario or provide targeted reinforcement training based on the trainees' real-time performance. Most systems only record the overall results, making it difficult to conduct multi-dimensional data analysis and attribution of techniques, strength, sequence, and verbal interactions with patients during the operation process. Summary of the Invention

[0004] (a) Technical problems to be solved In view of the above-mentioned shortcomings of the existing technology, the present invention provides an AI training system based on clinical scenario simulation, which can effectively solve the problems of the existing technology.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: This invention discloses an AI training system based on clinical scenario simulation, comprising: The data construction module is used to obtain the performance parameters and result parameters of historical clinical cases from the clinical database according to the preset clinical operation indicators, and form the original training dataset. The scenario modeling module provides a user interface and constructs a three-dimensional training model with multiple different clinical scenarios based on the original training dataset. The three-dimensional training model generates and projects instructional video or audio content corresponding to the selected scenario. The interactive perception module is used to capture in real time the motion images and voice feedback generated by the user in response to the selected situation through image and audio acquisition devices; the image acquisition devices are depth cameras or motion capture sensors, used to obtain the spatial coordinates and key point data of the user's movements; the audio acquisition devices are several microphone arrays. The feature extraction module is used to process the captured motion images and voice feedback, and extract key motion features that describe the form, location, and timing of the user's actions, as well as key voice features that describe the semantics, intonation, and keywords of the user's voice. The intelligent evaluation module is used to pre-train a state recognition model using reinforcement learning algorithms. The model takes key action features and key voice features as input, calculates the matching degree with the ideal state under the preset situation, and outputs a pass coefficient that represents the quality of the user's current comprehensive feedback. The decision-making module compares the pass coefficient with the preset pass threshold to determine whether the user feedback meets the requirements. The preset pass threshold is a configurable dynamic value that adaptively adjusts according to the complexity of the selected training scenario and the training stage. The deviation analysis module is used to trigger a step-by-step comparison between the key features of user feedback and the target features when the decision-making module determines that the user feedback does not meet the requirements. This helps to identify deviations in the direction or amplitude of the action, as well as semantic deviations in the voice content. The error correction feedback module is used to generate error correction graphic marks, including visual annotations of the location or amplitude of erroneous actions, and voice error correction marks, including prompts for erroneous voice content, based on the output of the deviation analysis module. The marks are then fed back to the user interface of the context modeling module, and the error correction mark data and subsequent user performance data are fed back to the interaction perception module and the intelligent evaluation module to achieve continuous optimization of model parameters and dynamic adjustment of the training context.

[0006] Furthermore, the preset operational indicators in the data construction module include clinical operation steps, time efficiency, and error rate; the performance parameters of the historical clinical cases include operation step sequence, operation time, instrument usage data, or vital sign monitoring data; and the result parameters include patient prognostic indicators, complication status, or operation success rate.

[0007] Furthermore, the scenario modeling module presets various clinical scenarios, including routine diagnosis and treatment procedures, emergency critical care, and doctor-patient communication scenarios, and the 3D training model supports dynamic switching of scenarios and selection of difficulty levels.

[0008] Furthermore, the extraction process of key action features in the feature extraction module includes: The continuous image sequence captured by the image acquisition device is denoised and enhanced to construct a three-dimensional skeletal model of the user's actions; Based on a 3D skeletal model, one or more keyframes describing the complete action unit are automatically identified and extracted according to the start, peak, and end points of the action. The three-dimensional skeleton model in the keyframe is calculated and quantized features are extracted. The quantized features include: the three-dimensional spatial coordinates of specific joints, the vector angle between adjacent joints, the smoothness of the movement trajectory, the cycle time or speed of the movement execution, and the standard orientation distance of the hands or instruments relative to the virtual anatomical structure. The quantitative features are combined with the standard procedure sequence of clinical operation to form an action feature vector, which serves as the key action feature output representing the user's action form, location, and timing.

[0009] Furthermore, the extraction process of key speech features in the feature extraction module includes: The user's voice feedback captured by the audio acquisition device is subjected to noise reduction and enhancement processing, and is converted into a corresponding text sequence using automatic speech recognition technology; Natural language processing analysis is performed on the text sequence to extract deep semantic features and obtain the accuracy of the identified medical terms or standard communication phrases, the relevance of the terms to the preset clinical context, the logical completeness of the statement content, and whether it contains key information items. The original speech signal is analyzed to extract paralinguistic features, which include: intonation, loudness, speech rate variation patterns, presence of inappropriate hesitation or interruption, and overall emotional tendency. The text sequence, semantic features, and paralinguistic features are fused and vectorized to form a speech feature vector, which serves as the key speech feature output representing the user's speech content, logic, and expression.

[0010] Furthermore, the state recognition model in the intelligent evaluation module adopts a deep Q-network model, taking the observed state composed of key action features and key speech features as input, and outputting the estimated value corresponding to all possible feedback actions performed in the current observed state. The construction process of the state recognition model includes the following steps: Step 51: Establish an experience replay memory bank to store training samples consisting of four tuples, which are generated by defining historical interaction data, the observation state, the action taken, the immediate reward obtained, and the new observation state. Step 52: Compare the key action features and key speech features extracted by the feature extraction module in the current training cycle with the target state features in the preset scenario, and use this feature difference vector as the input observation state of the current deep Q network model; Step 53: Initialize a current Q-network and a target Q-network with the same structure but delayed parameter updates; Step 54: Train the current Q-network using batch samples randomly sampled from the experience replay memory. Update the parameters of the current Q-network by calculating the temporal difference error between the target Q-value and the current Q-network output value as the loss function and minimizing this loss. The target Q-value is calculated as the sum of the maximum predicted value of the target Q-network for the next state and the immediate reward. Step 55: Periodically copy the parameters of the current Q network to the target Q network; Step 56: Iterate through steps 52 to 55 until the model converges.

[0011] Furthermore, the process for identifying the positional or amplitude deviation of the action in the deviation analysis module is as follows: Based on the current clinical context in the context modeling module, the three-dimensional spatial coordinate sequence of preset standard operating actions is extracted to form an action template; The image acquisition device in the interactive perception module captures continuous motion frames of the user, and uses skeletal tracking or human posture estimation algorithms to calculate the three-dimensional spatial coordinates of key body parts of the user and the equipment in real time. The real-time parsed user coordinate sequence is matched frame by frame or key frame with the action template to calculate the orientation deviation vector in three-dimensional space, the similarity of motion trajectory, the difference of joint angle, and the delay or advance of operation timing. When the magnitude, joint angle difference, trajectory similarity or temporal difference of the azimuth deviation vector do not meet the preset threshold, it is determined that there is a corresponding azimuth deviation or amplitude deviation.

[0012] Furthermore, the process of identifying semantic deviations in the speech content within the deviation analysis module is as follows: The speech text extracted by the feature extraction module is used to perform intent recognition and entity extraction using natural language processing technology to generate a semantic framework of the user's speech. Based on the current clinical context, a pre-set standardized communication script or key information list is invoked. The standard includes essential points for informing patients about their condition, medical terminology, clarity of instructions, and humanistic care language. Compare the user's semantic framework with the standard specifications to identify whether there are missing information, misuse of terminology, logical order errors or potential ambiguities; The paralinguistic features of speech, including speech rate, pauses, and tone of voice, are analyzed and compared with the ideal expression pattern for the context. When there are errors in semantic logic or the expression features deviate from the ideal pattern beyond the allowable range, it is determined that there is a semantic deviation or normative deviation.

[0013] Furthermore, the error correction graphic markers generated by the error correction feedback module are correction illustrations superimposed on the real-time video screen, such as arrows, highlighted boxes, or virtual skeleton lines; the generated voice error correction markers are voice prompts or correct voice example playback.

[0014] Furthermore, the data construction module and the context modeling module are interconnected via a wireless network; the interactive perception module, the context modeling module, the feature extraction module, and the intelligent evaluation module are interconnected via a wireless network; the decision-making module, the intelligent evaluation module, and the deviation analysis module are interconnected via a wireless network; and the deviation analysis module and the error correction feedback module are interconnected via a wireless network.

[0015] (III) Beneficial Effects Compared with the known prior art, the technical solution provided by this invention has the following beneficial effects: 1. By constructing a state recognition model, the system replaces the traditional assessment mode that relies on subjective human judgment. The system can collect students' action and voice data in real time and immediately convert them into quantifiable pass coefficients and specific deviation analyses. This greatly reduces the burden on instructors and ensures the objectivity and consistency of assessment standards, enabling students to recognize error details as soon as possible and accelerating the formation of correct skill patterns.

[0016] 2. Through 3D modeling and a pre-set scenario library, diverse training scenarios can be generated, ranging from routine operations to extreme critical cases. This breaks the limitations of physical simulation equipment in terms of space, time, and cost. Training content can be dynamically selected and its difficulty adjusted according to training objectives and trainee levels, achieving personalized training. This greatly enhances the realism of the training and trainee participation, effectively improving their clinical decision-making and adaptability under high-pressure environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the overall framework of the present invention.

[0019] The labels in the diagram represent: 1. Data Construction Module; 2. Context Modeling Module; 3. Interaction Perception Module; 4. Feature Extraction Module; 5. Intelligent Evaluation Module; 6. Decision Judgment Module; 7. Deviation Analysis Module; 8. Error Correction Feedback Module. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] The present invention will be further described below with reference to embodiments.

[0022] The AI ​​training system based on clinical scenario simulation in this embodiment, such as Figure 1 As shown, it includes: Data construction module 1 is used to obtain the performance parameters and outcome parameters of historical clinical cases from the clinical database according to the preset clinical operation indicators, and form the original training dataset. The preset operation indicators include clinical operation steps, time efficiency and error rate. The performance parameters of historical clinical cases include operation step sequence, operation time, instrument usage data or vital sign monitoring data. The outcome parameters include patient prognostic indicators, complication status or operation success rate.

[0023] Context modeling module 2 provides a user interface and constructs a 3D training model with multiple different clinical scenarios based on the original training dataset. The 3D training model generates and projects guidance video or audio content corresponding to the selected scenario. The multiple preset clinical scenarios include routine diagnosis and treatment procedures, emergency critical care, and doctor-patient communication scenarios. The 3D training model supports dynamic switching of scenarios and selection of difficulty levels.

[0024] The interactive perception module 3 is used to capture in real time the motion images and voice feedback generated by the user in response to the selected situation through image and audio acquisition devices; the image acquisition device is a depth camera or motion capture sensor, used to obtain the spatial coordinates and key point data of the user's actions; the audio acquisition device is a microphone array.

[0025] Feature extraction module 4 is used to process the captured motion images and voice feedback, extracting key motion features describing the form, location, and timing of user actions, as well as key voice features describing the semantics, intonation, and keywords of user speech; the key motion feature extraction process includes: The continuous image sequence captured by the image acquisition device is denoised and enhanced to construct a three-dimensional skeletal model of the user's actions; Based on a 3D skeletal model, one or more keyframes describing the complete action unit are automatically identified and extracted according to the start, peak, and end points of the action. The 3D skeleton model in the keyframe is calculated and quantized features are extracted. The quantized features include: the 3D spatial coordinates of specific joints, the vector angle between adjacent joints, the smoothness of the motion trajectory, the cycle time or speed of the motion execution, and the standard orientation distance of the hands or instruments relative to the virtual anatomical structure. The quantitative features are combined with the standard procedure sequence of clinical operation to form an action feature vector, which serves as the key action feature output representing the user's action form, orientation, and timing. The process of extracting key speech features includes: The user's voice feedback captured by the audio acquisition device is subjected to noise reduction and enhancement processing, and then converted into a corresponding text sequence using automatic speech recognition technology; Natural language processing analysis is performed on the text sequence to extract deep semantic features and obtain the accuracy of the identified medical terms or standard communication phrases, the relevance of the terms to the preset clinical context, the logical completeness of the statements, and whether they contain key information items. The original speech signal is analyzed to extract paralinguistic features, which include: intonation, loudness, speech rate variation patterns, presence of inappropriate hesitation or interruption, and overall emotional tendency. By fusing and vectorizing text sequences, semantic features, and paralinguistic features, a speech feature vector is formed, which serves as the key speech feature output representing the user's speech content, logic, and expression.

[0026] The intelligent evaluation module 5 is used to pre-train a state recognition model using a reinforcement learning algorithm. The model takes key action features and key speech features as input, calculates the matching degree with the ideal state under a preset situation, and outputs a pass coefficient that represents the quality of the user's current comprehensive feedback.

[0027] The decision-making module 6 is used to compare the pass coefficient with the preset pass threshold to determine whether the user feedback meets the requirements. The preset pass threshold is a configurable dynamic value that is adaptively adjusted according to the complexity of the selected training scenario and the training stage.

[0028] The deviation analysis module 7 is used to trigger a step-by-step comparison between the key features of user feedback and the target features when the decision-making module 6 determines that the feedback does not meet the requirements. This comparison identifies deviations in the direction or amplitude of the action, as well as semantic deviations in the speech content. The process for identifying deviations in the direction or amplitude of the action is as follows: Based on the current clinical situation in Context Modeling Module 2, the three-dimensional spatial coordinate sequence of preset standard operating actions is extracted to form an action template; The image acquisition device in the interactive perception module 3 captures continuous motion frames of the user, and uses skeletal tracking or human posture estimation algorithms to calculate the three-dimensional spatial coordinates of key body parts of the user and the equipment in real time. The real-time parsed user coordinate sequence is matched frame by frame or key frame with the action template to calculate the orientation deviation vector in three-dimensional space, the similarity of motion trajectory, the difference of joint angle, and the delay or advance of operation timing. When the magnitude, joint angle difference, trajectory similarity or temporal difference of the azimuth deviation vector do not meet the preset threshold, it is determined that there is a corresponding azimuth deviation or amplitude deviation. By solving continuous motions into a three-dimensional coordinate sequence and comparing it frame by frame with a standard template, the system can not only identify positional errors, such as deviation of instrument contact points, but also quantify abnormal postures, such as joint angles exceeding limits and improper rhythm, such as incorrect operation steps or speed. This solves the pain points of traditional assessments, which are highly subjective and difficult to capture subtle dynamic flaws, and provides an objective real-time feedback basis for clinical skills training.

[0029] The process of recognizing semantic deviations in speech content is as follows: The speech text extracted by feature extraction module 4 is subjected to intent recognition and entity extraction using natural language processing technology to generate a semantic framework of the user's speech. Based on the current clinical context, a pre-set standardized communication script or key information list is invoked, which includes essential points for informing patients about their condition, medical terminology, clarity of instructions, and humanistic care language. Compare the user's semantic framework with the standard specifications to identify whether there are missing information, misuse of terminology, logical order errors or potential ambiguities; Analyze the paralinguistic features of speech, including speech rate, pauses, and tone of voice, and compare them with the ideal expression pattern for the context. When there are errors in semantic logic or the expression features deviate from the ideal pattern beyond the allowable range, it is determined that there is a semantic deviation or normative deviation. By constructing a semantic framework and comparing it with contextualized standard scripts, the system can identify semantic-level problems such as missing information, misuse of terminology, and logical sequence errors. Combined with the analysis of paralinguistic features such as speech rate and pauses, the system can also assess the standardization of expression such as humanistic care and communication efficiency. This effectively solves the problem that traditional methods cannot assess communication quality and improves the effectiveness of communication skills training.

[0030] The error correction feedback module 8 is used to generate error correction graphic markers, including visual annotations of the location or amplitude of erroneous actions, and voice error correction markers, including prompts for erroneous voice content, based on the output of the deviation analysis module 7. The generated error correction graphic markers are correction illustrations superimposed on the real-time video screen, such as arrows, highlighted boxes, or virtual skeleton lines; the generated voice error correction markers are voice prompts or playback of correct voice examples. The error correction marker data and subsequent user performance data are fed back to the interaction perception module 3 and the intelligent evaluation module 5 to achieve continuous optimization of model parameters and dynamic adjustment of the training context.

[0031] Data construction module 1 and context modeling module 2 are connected via a wireless network. Interactive perception module 3, context modeling module 2, feature extraction module 4, and intelligent evaluation module 5 are connected via a wireless network. Decision judgment module 6, intelligent evaluation module 5, and deviation analysis module 7 are connected via a wireless network. Deviation analysis module 7 and error correction feedback module 8 are connected via a wireless network.

[0032] Compared with existing technologies, this embodiment generates dynamic clinical scenarios based on real historical data, and captures and recognizes user feedback simultaneously through images and voice. It uses a state recognition model to perform real-time quantitative assessment and precise deviation analysis, generating specific action and voice error correction marks and providing immediate personalized guidance. This surpasses the limitations of traditional simulation training, which relies on manual observation, has delayed feedback, and is highly subjective, thus achieving immersive clinical skills training.

[0033] At other levels, this embodiment provides a state recognition model. The state recognition model adopts a deep Q-network model, taking the observed state composed of key action features and key speech features as input, and outputting the estimated value corresponding to all possible feedback actions performed in the current observed state. The construction process of the state recognition model includes the following steps: Step 51: Establish an experience replay memory bank to store training samples consisting of four tuples, which are generated by defining historical interaction data, the observation state, the action taken, the immediate reward obtained, and the new observation state. Step 52: Compare the key action features and key speech features extracted by the feature extraction module 4 in the current training cycle with the target state features in the preset scenario, and use this feature difference vector as the input observation state of the current deep Q network model. Step 53: Initialize a current Q-network and a target Q-network with the same structure but delayed parameter updates; Step 54: Train the current Q-network using batch samples randomly sampled from the experience replay memory. Update the parameters of the current Q-network by calculating the temporal difference error between the target Q-value and the current Q-network output value as the loss function and minimizing this loss. The target Q-value is calculated as the sum of the maximum predicted value of the target Q-network for the next state and the immediate reward. Step 55: Periodically copy the parameters of the current Q network to the target Q network; Step 56: Iterative training from Step 52 to Step 55 continues until the model converges; wherein, when deploying the application, the pass coefficient output by the intelligent evaluation module 5 is generated by the aggregated calculation of the value estimates of all actions defined as correct feedback under the current observation state by the trained current Q network, and the maximum value or weighted average is taken. This embodiment introduces a reinforcement learning framework based on deep Q-networks, directly mapping the difference between clinical operational characteristics and preset target states into a learnable value assessment problem. The model can autonomously optimize its assessment strategy through continuous interaction with the environment and guidance from reward signals, thereby achieving adaptive dynamic quantitative assessment of clinical competence performance and improving the intelligence of the AI ​​training system.

[0034] In summary, this invention provides highly immersive and diverse simulated scenarios by constructing a three-dimensional training model, greatly expanding the scenario coverage and realism of traditional training. Through the simultaneous acquisition and feature extraction of images and voice, it can quantify trainees' operational behaviors and communication content. The state recognition model built based on reinforcement learning replaces the traditional subjective evaluation that relies on manual observation, providing real-time feedback on pass / fail coefficients. When a failure is determined, the system does not only give the result, but can accurately locate the error details through intelligent deviation analysis and generate concrete audiovisual error correction marks, such as visual guidance arrows or voice examples, thereby achieving personalized tutoring. This invention not only accelerates the skill mastery curve and reduces the risks and costs of practicing on real patients, but also provides a reliable technical foundation for large-scale clinical skills assessment and certification through standardized intelligent evaluation.

[0035] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AI training system based on clinical scenario simulation, characterized in that, include: The data construction module is used to obtain the performance parameters and result parameters of historical clinical cases from the clinical database according to the preset clinical operation indicators, and form the original training dataset. The scenario modeling module provides a user interface and constructs a three-dimensional training model with multiple different clinical scenarios based on the original training dataset. The three-dimensional training model generates and projects instructional video or audio content corresponding to the selected scenario. The interactive perception module is used to capture, in real time, the motion images and voice feedback generated by the user in response to the selected situation through image and audio acquisition devices; The feature extraction module is used to process the captured motion images and voice feedback, and extract key motion features that describe the form, location, and timing of the user's actions, as well as key voice features that describe the semantics, intonation, and keywords of the user's voice. The intelligent evaluation module is used to pre-train a state recognition model using reinforcement learning algorithms. The model takes key action features and key voice features as input, calculates the matching degree with the ideal state under the preset situation, and outputs a pass coefficient that represents the quality of the user's current comprehensive feedback. The decision-making module is used to compare the pass coefficient with the preset pass threshold to determine whether the user feedback meets the requirements. The deviation analysis module is used to trigger a step-by-step comparison between the key features of user feedback and the target features when the decision-making module determines that the user feedback does not meet the requirements. This helps to identify deviations in the direction or amplitude of the action, as well as semantic deviations in the voice content. The error correction feedback module is used to generate error correction graphic marks, including visual annotations of the location or amplitude of the erroneous action, and voice error correction marks, including prompts for erroneous voice content, based on the output of the deviation analysis module, and feed the marks back to the user interface of the context modeling module.

2. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The preset operational indicators in the data construction module include clinical operation steps, time efficiency, and error rate; the performance parameters of the historical clinical cases include operation step sequence, operation time, instrument usage data, or vital sign monitoring data; and the result parameters include patient prognostic indicators, complication status, or operation success rate.

3. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The scenario modeling module presets various clinical scenarios, including routine diagnosis and treatment procedures, emergency critical care, and doctor-patient communication scenarios. The 3D training model supports dynamic switching of scenarios and selection of difficulty levels.

4. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The extraction process of key action features in the feature extraction module includes: The continuous image sequence captured by the image acquisition device is denoised and enhanced to construct a three-dimensional skeletal model of the user's actions; Based on a 3D skeletal model, one or more keyframes describing the complete action unit are automatically identified and extracted according to the start, peak, and end points of the action. The three-dimensional skeleton model in the keyframe is calculated and quantized features are extracted. The quantized features include: the three-dimensional spatial coordinates of specific joints, the vector angle between adjacent joints, the smoothness of the movement trajectory, the cycle time or speed of the movement execution, and the standard orientation distance of the hands or instruments relative to the virtual anatomical structure. The quantitative features are combined with the standard procedure sequence of clinical operations to encode action feature vectors, which are then output as key action features.

5. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The extraction process of key speech features in the feature extraction module includes: The user's voice feedback captured by the audio acquisition device is subjected to noise reduction and enhancement processing, and is converted into a corresponding text sequence using automatic speech recognition technology; Natural language processing analysis is performed on the text sequence to extract deep semantic features and obtain the accuracy of the identified medical terms or standard communication phrases, the relevance of the terms to the preset clinical context, the logical completeness of the statement content, and whether it contains key information items. The original speech signal is analyzed to extract paralinguistic features, which include: intonation, loudness, speech rate variation patterns, presence of inappropriate hesitation or interruption, and overall emotional tendency. The text sequence, semantic features, and paralinguistic features are fused and vectorized to form a speech feature vector, which is then output as the key speech feature.

6. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The state recognition model in the intelligent evaluation module adopts a deep Q-network model. It takes the observed state, which is composed of key action features and key speech features, as input, and outputs the estimated value corresponding to all possible feedback actions performed in the current observed state. The construction process of the state recognition model includes the following steps: Step 51: Establish an experience replay memory bank to store training samples consisting of four tuples, which are generated by defining historical interaction data, the observation state, the action taken, the immediate reward obtained, and the new observation state. Step 52: Compare the key action features and key speech features extracted by the feature extraction module in the current training cycle with the target state features in the preset scenario, and use this feature difference vector as the input observation state of the current deep Q network model; Step 53: Initialize a current Q-network and a target Q-network with the same structure but delayed parameter updates; Step 54: Train the current Q-network using batch samples randomly sampled from the experience replay memory. Update the parameters of the current Q-network by calculating the temporal difference error between the target Q-value and the current Q-network output value as the loss function and minimizing this loss. Step 55: Periodically copy the parameters of the current Q network to the target Q network; Step 56: Iterate through steps 52 to 55 until the model converges.

7. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The process for identifying the position or amplitude deviation of the action in the deviation analysis module is as follows: Based on the current clinical context in the context modeling module, the three-dimensional spatial coordinate sequence of preset standard operating actions is extracted to form an action template; The image acquisition device in the interactive perception module captures continuous motion frames of the user, and uses skeletal tracking or human posture estimation algorithms to calculate the three-dimensional spatial coordinates of key body parts of the user and the equipment in real time. The real-time parsed user coordinate sequence is matched frame by frame or key frame with the action template to calculate the orientation deviation vector in three-dimensional space, the similarity of motion trajectory, the difference of joint angle, and the delay or advance of operation timing. When the magnitude, joint angle difference, trajectory similarity or temporal difference of the azimuth deviation vector do not meet the preset threshold, it is determined that there is a corresponding azimuth deviation or amplitude deviation.

8. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The process of identifying semantic deviations in speech content in the deviation analysis module is as follows: The speech text extracted by the feature extraction module is used to perform intent recognition and entity extraction using natural language processing technology to generate a semantic framework of the user's speech. Based on the current clinical context, invoke the pre-set standardized communication script or key information list; Compare the user's semantic framework with the standard specifications to identify whether there are missing information, misuse of terminology, logical order errors or potential ambiguities; The paralinguistic features of speech, including speech rate, pauses, and tone of voice, are analyzed and compared with the ideal expression pattern for the context. When there is an error in the semantic logic or the expression features deviate from the ideal pattern beyond the allowable range, a semantic deviation is determined.

9. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The error correction feedback module generates error correction graphic markers that are arrows, highlight boxes, or virtual skeleton lines superimposed on the real-time video screen as correction illustrations; the generated voice error correction markers are voice prompts or correct voice example playback.

10. The AI ​​training system based on clinical scenario simulation according to claim 1, characterized in that, The data construction module and the context modeling module are interconnected via a wireless network. The interactive perception module, the context modeling module, the feature extraction module, and the intelligent evaluation module are interconnected via a wireless network. The decision-making module, the intelligent evaluation module, and the deviation analysis module are interconnected via a wireless network. The deviation analysis module and the error correction feedback module are interconnected via a wireless network.

Images