Comprehensive scene training system and method for cardio-pulmonary resuscitation

Abstract

The invention discloses a comprehensive scene training system and method for cardio-pulmonary resuscitation, and particularly relates to the technical field of first-aid training, the synchronous deviation of student operation data and virtual feedback data is evaluated through a hidden variable Markov model, and three-dimensional rendering analytic hierarchy process is carried out by combining optical flow analysis with a deep convolutional generative adversarial network, so that the comprehensive scene training system and method for cardio-pulmonary resuscitation are obtained. The visual penetrability and the multi-view consistency of the object are ensured; besides, a sparse coding network and a gradient adaptive comparison algorithm are used for extracting pathological feature images and evaluating visual deviation between visual angles, so that the accuracy of virtual pathological display is optimized, immersion and operation effectiveness of cardiopulmonary resuscitation first-aid training are effectively improved, and a more real and accurate cardiopulmonary resuscitation training environment is provided for trainees.

Description

Technical Field

[0001] The present invention relates to the technical field of first aid training, and more particularly to a comprehensive scenario training system and method for cardiopulmonary resuscitation. Background Art

[0002] In the existing technology, first aid training has gradually introduced 3D interaction and virtual reality technology to enhance the immersion and practical effect of training. Through the interactive teaching platform built by 3D Unity, combined with interactive projection and real-time data communication, a real first aid scene simulation is realized. First aid training includes virtual training systems, cardiopulmonary resuscitation and trauma first aid simulators, etc., supports multi-scene simulations such as city streets and hospital halls, and uses high-definition projection to display on the classroom wall to create a complete on-site environment. The first aid scene is linked to the simulator to display vital signs in real time. Students can perform first aid operations such as external cardiac compression and artificial ventilation according to the changes in the virtual patient's condition, allowing students to practice complex first aid skills in the virtual environment of the virtual training system.

[0003] In the virtual training system, if the position of objects in the virtual scene does not accurately correspond to the physical equipment in the real environment, it will cause spatial dislocation during cardiopulmonary resuscitation training operations, affecting the training effect of cardiopulmonary resuscitation training.

[0004] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present invention provide a comprehensive scenario training system and method for cardiopulmonary resuscitation to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A comprehensive scenario training method for cardiopulmonary resuscitation, comprising the following steps:

[0008] The students’ operation data and virtual feedback data were analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback.

[0009] Analyze the synchronization deviation rate between the physiological animation and the physical feedback within a preset interval to determine whether the synchronization between the physiological animation and the physical feedback is abnormal;

[0010] When the synchronization between physiological animation and physical feedback is abnormal, correct the synchronization deviation;

[0011] When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer; the pathological feature images of each perspective are extracted through a sparse coding network, and the severity of visual deviation between perspectives is evaluated using a gradient adaptive contrast algorithm;

[0012] Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it is determined whether the accuracy risk of the virtual pathology display is within the allowable range.

[0013] In a preferred embodiment, the trainee operation data and virtual feedback data are analyzed by a hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback, specifically:

[0014] Acquire the trainee operation data and corresponding virtual feedback data during the cardiopulmonary resuscitation training, and pre-process the trainee operation data and virtual feedback data;

[0015] Based on the pre-processed student operation data and virtual feedback data, a hidden variable Markov model is constructed; the state space, observation sequence and transition probability matrix of the hidden variable Markov model are clarified;

[0016] The expectation maximization algorithm is used to train the parameters of the hidden variable Markov model, and the optimal parameters of the hidden variable Markov model are solved through iterative optimization;

[0017] Using the trained hidden variable Markov model, the trainees' operation data and virtual feedback data were sequence analyzed to calculate the synchronization deviation rate between physiological animation and physical feedback.

[0018] In a preferred embodiment, the synchronization deviation rate between the physiological animation and the physical feedback within a preset interval is analyzed to determine whether the synchronization between the physiological animation and the physical feedback is abnormal, specifically:

[0019] Obtaining an average value of the overall synchronization deviation rate within a preset interval, and marking the average value of the overall synchronization deviation rate within the preset interval as a synchronization deviation index;

[0020] Set the upper and lower thresholds of synchronization deviation;

[0021] When the synchronization deviation index is between the synchronization deviation upper limit threshold and the synchronization deviation lower limit threshold, it is determined that the synchronization between the physiological animation and the physical feedback is normal; otherwise, it is determined that the synchronization between the physiological animation and the physical feedback is abnormal.

[0022] In a preferred embodiment, when the synchronization between the physiological animation and the physical feedback is abnormal, the synchronization deviation is corrected, specifically:

[0023] Analyze the synchronization deviation between physiological animation and physical feedback, and determine the direction of the deviation;

[0024] The response time of the virtual feedback is appropriately adjusted according to the deviation type;

[0025] Adjust the update frequency of virtual feedback;

[0026] Monitor synchronization in real time during the correction process and dynamically adjust response parameters.

[0027] In a preferred embodiment, optical flow analysis is combined with deep convolutional generative adversarial networks to analyze 3D rendering layer information and evaluate the visual penetration of objects in each layer, specifically:

[0028] Get the current 3D rendering layer information, including the spatial position, texture features and structural relationships of all objects in the scene;

[0029] The optical flow algorithm is used to process the 3D rendering layer information of continuous frames to generate the optical flow field features of each layer of objects in the time series;

[0030] A deep convolutional generative adversarial network model is constructed based on the extracted optical flow field features to learn the visual features and hierarchical relationships of each layer of rendering information in the three-dimensional scene;

[0031] Based on the deep convolutional generative adversarial network model, the penetration analysis of the 3D rendering layer information is performed to evaluate the visual penetration of objects in each layer.

[0032] In a preferred embodiment, the penetration analysis of the three-dimensional rendering layer information is performed based on the deep convolutional generative adversarial network model to evaluate the visual penetration of objects in each layer, specifically:

[0033] Calculate the visual penetration coefficient, the calculation formula is: Among them, P represents the visual penetration coefficient, n is the total number of rendering layers, and d k The k-th layer rendering depth value extracted by the deep convolutional generative adversarial network, r k is the true depth value of the object in the kth layer, where k is the number of the rendering layer.

[0034] In a preferred embodiment, the pathological feature images of each viewing angle are extracted by a sparse coding network, and the severity of the visual deviation between viewing angles is evaluated by a gradient adaptive contrast algorithm, specifically:

[0035] Acquire images of the same pathological scene from different perspectives and calibrate the imaging parameters of each image;

[0036] The images of each view are processed through a sparse coding network to extract and encode the key pathological feature information of each view image and generate an independent feature map representation;

[0037] The feature maps of images from different viewpoints are input into the feature association model, the corresponding relationship of the pathological features in the images from each viewpoint is analyzed, and a feature comparison vector across viewpoints is generated;

[0038] Based on the feature contrast vector, a gradient adaptive contrast algorithm is applied to gradually calculate the difference amplitude of pathological features between different viewing angles;

[0039] The difference in pathological features between different viewing angles was summarized and calculated to quantify the severity of visual deviation between viewing angles.

[0040] In a preferred embodiment, a gradient adaptive contrast algorithm is applied based on the feature contrast vector to gradually calculate the difference amplitude of pathological features between different viewing angles, specifically:

[0041] The gradient adaptive contrast algorithm is applied to calculate the feature difference of each pair of perspectives, and the feature difference amplitude index is calculated, which is expressed as: Among them, A qp is the feature difference magnitude index, F q，r and F p，r F q and F p The rth component of , c represents the dimension of the eigenvector, r represents the number of the component, ||V qp ||Euclidean distance representing feature differences, F q represents the feature vector of the qth view, F p represents the feature vector of the pth view, q represents the number of the first view, and p represents the number of the second view.

[0042] In a preferred embodiment, based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviation between viewing angles, it is determined whether the accuracy risk of the virtual pathology display is within an allowable range, specifically:

[0043] Set the threshold of the visual penetration coefficient, denoted as α; set the threshold of the visual deviation index, denoted as β;

[0044] If the visual penetration coefficient is less than or equal to α, it is judged that the visual penetration of objects in each layer is within a reasonable range; if the visual penetration coefficient is greater than α, it is judged that there is a high risk of layer confusion;

[0045] If the visual deviation index is less than or equal to β, it is judged that the visual deviation between the perspectives is within the allowable range; if the visual deviation index is greater than β, it is judged that the consistency between the perspectives is poor;

[0046] When the visual penetration of objects in each layer is within a reasonable range and the visual deviation between viewing angles is within an allowable range, the accuracy risk of the virtual pathology display is determined to be within an allowable range.

[0047] On the other hand, the present invention provides a comprehensive scenario training system for cardiopulmonary resuscitation, including a synchronization deviation assessment module, a synchronization anomaly detection module, a synchronization deviation correction module, a visual penetration analysis module, a visual deviation assessment module, and a display accuracy judgment module;

[0048] Synchronization deviation evaluation module: Analyze the trainee operation data and virtual feedback data through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback;

[0049] Synchronization anomaly detection module: analyzes the synchronization deviation rate between physiological animation and physical feedback within a preset interval to determine whether the synchronization between physiological animation and physical feedback is abnormal;

[0050] Synchronization deviation correction module: when the synchronization between physiological animation and physical feedback is abnormal, the synchronization deviation is corrected;

[0051] Visual penetration analysis module: When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer;

[0052] Visual deviation assessment module: When the synchronization between physiological animation and physical feedback is normal, the pathological feature images of each view are extracted through a sparse coding network, and the severity of visual deviation between viewpoints is assessed using a gradient adaptive contrast algorithm;

[0053] Display accuracy judgment module: Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it determines whether the accuracy risk of the virtual pathology display is within the allowable range.

[0054] The technical effects and advantages of the comprehensive scenario training system and method for cardiopulmonary resuscitation of the present invention are as follows:

[0055] 1. The present invention makes the analysis of trainee operation data and virtual feedback data more accurate through the comprehensive application of advanced technologies such as hidden variable Markov model, optical flow analysis, deep convolution generative adversarial network, sparse coding network and gradient adaptive contrast algorithm, which greatly improves the synchronization accuracy of physiological animation and physical feedback. This synchronous evaluation and adjustment mechanism ensures that what the trainee sees is what he gets during the operation, avoids the inconsistency between physiological reaction and operation feedback in the virtual scene, makes the operation process more in line with the needs of real emergency situations, and can achieve high-precision operation and feedback linkage even in complex virtual cardiopulmonary resuscitation emergency scenes, which helps to improve the realism of training and the trainees' confidence in operation.

[0056] 2. By introducing optical flow analysis and deep convolution generative adversarial network models to evaluate the hierarchical information of three-dimensional rendering, the present invention effectively improves the visual penetration performance of objects in each layer and the visual consistency of multiple perspectives, so that students can clearly distinguish different levels and spatial positions when observing virtual pathology displays from multiple perspectives, reducing the risk of visual confusion. At the same time, the sparse coding network is combined with the gradient adaptive contrast algorithm to evaluate the degree of visual deviation between perspectives, further ensuring the consistency of the performance of virtual pathological features under different perspectives. This comprehensive accuracy assessment and adjustment method makes the pathology display of the virtual training system more consistent with the real observation scenario, thereby significantly improving the effect of cardiopulmonary resuscitation training, helping students to master operating skills more accurately and improve first aid skills. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a comprehensive scenario training method for cardiopulmonary resuscitation according to the present invention;

[0058] Figure 2 The present invention is a structural schematic diagram of a comprehensive scenario training system for cardiopulmonary resuscitation. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Example 1

[0061] Figure 1 The present invention provides a comprehensive scenario training method for cardiopulmonary resuscitation, which comprises the following steps:

[0062] The students' operation data and virtual feedback data are analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback.

[0063] The synchronization deviation rate between the physiological animation and the physical feedback within the preset interval is analyzed to determine whether the synchronization between the physiological animation and the physical feedback is abnormal.

[0064] When the synchronization between physiological animation and physical feedback is abnormal, correct the synchronization deviation.

[0065] When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer. The pathological feature images of each perspective are extracted through a sparse coding network, and the severity of visual deviation between perspectives is evaluated using a gradient adaptive contrast algorithm.

[0066] Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it is determined whether the accuracy risk of the virtual pathology display is within the allowable range.

[0067] The students’ operation data and virtual feedback data were analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback. Specifically:

[0068] Obtain the trainee operation data and corresponding virtual feedback data during the cardiopulmonary resuscitation training, and pre-process the trainee operation data and virtual feedback data:

[0069] During the CPR training, the trainees’ operation data and corresponding virtual feedback data are collected in real time through sensors and interface modules. The trainees’ operation data include information such as compression force, compression frequency, and operation position. These data record the specific operation details of the trainees in CPR training. The virtual feedback data includes the vital signs information of the virtual patient as the trainees operate, such as the virtual patient’s heart rate changes, blood oxygen level, chest movement amplitude, etc. These feedback data can accurately reflect the dynamic changes of the virtual patient’s state by comparing with the operation data.

[0070] The preprocessing process of trainee operation data and virtual feedback data includes the following specific steps:

[0071] Remove noise data and outliers generated during the acquisition process. For example, for the force information in the operation data, if the values ​​of some sampling points are obviously deviated from the normal range, these data are regarded as noise or outliers and will be removed to maintain the authenticity and validity of the data.

[0072] All collected raw data is converted into a structured data format for subsequent modeling. Here, the operation data and feedback data are processed into time series data sets, where each data point corresponds to a time step, ensuring that the model can continuously use this data for analysis.

[0073] Since the acquisition rates of the trainee operation data and the virtual feedback data may be different, the time steps of the two data sources need to be synchronized to ensure that the data of the two are aligned and consistent on the same time axis.

[0074] Based on the preprocessed student operation data and virtual feedback data, a hidden variable Markov model is constructed; the state space, observation sequence and transition probability matrix of the hidden variable Markov model are clarified to describe the potential relationship between student operation and virtual feedback:

[0075] The hidden variable Markov model describes the potential relationship between students' operations and virtual feedback through state transition, which specifically includes the construction of hidden state space, observation sequence and transition probability matrix.

[0076] Define the state space: The state space of the hidden variable Markov model defines the set of internal states of the model. Each state represents a synchronization relationship between the student's operation and the virtual feedback. Assume that the state space contains a finite set of states, each state corresponds to a certain synchronization relationship between the student's operation and the virtual feedback.

[0077] Definition of observation sequence: The observation sequence represents the feature sequence extracted from the trainee's operation data and virtual feedback data. Each observation value represents the trainee's operation feature (such as pressing force and pressing position) and the corresponding virtual feedback (such as chest displacement and heartbeat response) at each time step. The observation sequence is a collection of specific observation data at different time steps in the time series.

[0078] Define the transition probability matrix: The state transition probability matrix of the hidden variable Markov model describes the transition probability between different states, that is, the probability of transitioning from one state to another. This matrix reflects the transition rules between the student's operation data and the virtual feedback data, and is used to predict how the current state changes to the next state.

[0079] Define initial probability vector: Define the initial probability vector, which represents the probability of the model being in each state in the initial time step. The initial probability vector reflects the level of synchronization between the learner and the virtual feedback under the initial operation.

[0080] The expectation maximization algorithm is used to train the parameters of the hidden variable Markov model, and the optimal parameters of the hidden variable Markov model are solved through iterative optimization:

[0081] In each iteration, the expected likelihood of the observation sequence is calculated based on the current parameter estimates. Specifically, the forward and backward algorithms are used to calculate the forward and backward probabilities respectively. The forward probability represents the probability of being in a certain state at a certain time step and observing part of the observation sequence; the backward probability represents the probability of being in a certain state at a certain time step and observing the sequence from the current step to the end of the sequence. The joint probability of the model being in a certain state at each time step is calculated by multiplying the forward and backward probabilities.

[0082] After the expectation step is completed, the model parameters (i.e., state transition probabilities, observation probabilities, and initial state probabilities) are adjusted to maximize the likelihood function. The new model parameter estimates are used for the next round of iterations until the model parameters converge.

[0083] Through multiple iterations of expectation and maximization steps, the model parameters are gradually optimized until convergence, completing parameter training. At this point, the model can better describe the potential relationship between the trainee's operation data and virtual feedback data, providing a basis for the calculation of the synchronization deviation rate.

[0084] Using the trained hidden variable Markov model, the trainees’ operation data and virtual feedback data are sequenced and the synchronization deviation rate between physiological animation and physical feedback is calculated:

[0085] The forward-backward algorithm is used to calculate the hidden state probability distribution of each time step. The hidden state sequence reflects the synchronization between the student's operation and the virtual feedback at each time step. Based on the hidden state sequence, the synchronization deviation between the student's operation and the virtual feedback at each time step is analyzed.

[0086] Calculate the overall synchronization deviation rate, the expression is: Where D represents the overall synchronization deviation rate, T represents the total number of time steps observed, and d t Represents the deviation between the student's operation data and the virtual feedback data at time step t.

[0087] The smaller the overall synchronization deviation rate is, the better the synchronization between the trainee's operation data and the virtual feedback data is.

[0088] The synchronization deviation rate between the physiological animation and the physical feedback within the preset interval is analyzed to determine whether the synchronization between the physiological animation and the physical feedback is abnormal, specifically:

[0089] Physiological animation is the physiological response of the virtual patient displayed on the screen as the trainee operates, such as heartbeat, breathing, chest rise and fall, etc. Physical feedback is the actual operation data collected in real time by sensors when the trainee applies operations (such as compression and ventilation) to the simulator during first aid training, such as strength, frequency, depth, etc.

[0090] An average value of the overall synchronization deviation rate within a preset interval is obtained, and the average value of the overall synchronization deviation rate within the preset interval is marked as a synchronization deviation index.

[0091] Among them, the preset interval refers to a sequence of operation and feedback data within a specific training time range, which is used to observe and analyze the synchronization between trainees' operations and virtual feedback within this interval, ensuring that the system's synchronization deviation can be evaluated within a limited time.

[0092] Set the synchronization deviation upper and lower thresholds.

[0093] When the synchronization deviation index is between the synchronization deviation upper limit threshold and the synchronization deviation lower limit threshold, it is determined that the synchronization between the physiological animation and the physical feedback is normal; otherwise, it is determined that the synchronization between the physiological animation and the physical feedback is abnormal.

[0094] Among them, the upper and lower thresholds of synchronization deviation are determined based on experimental data, training standards, and the tolerance range of real physiological responses. The upper threshold represents the maximum feedback delay allowed to avoid excessive lag in physiological animation. The lower threshold is set to a reasonable minimum deviation range to ensure that the feedback is not too idealized or mechanized. This range is consistent with the actual physiological response characteristics and meets the realism requirements of first aid training. Being lower than the lower threshold means that the physiological animation is too synchronized with the physical feedback and lacks real physiological delay, which may cause trainees to mistakenly believe that the physiological response in first aid will be completely timely. Such excessive synchronization will weaken the realism of the training, which is not conducive to trainees adapting to the natural feedback lag and strain requirements in the actual first aid environment, and affects their ability to respond to emergencies.

[0095] When the synchronization between physiological animation and physical feedback is abnormal, correct the synchronization deviation, specifically:

[0096] Analyze the synchronization deviation between physiological animation and physical feedback, and determine the direction of the deviation:

[0097] If the synchronization deviation index is higher than the synchronization deviation upper threshold, it means that there is a lag deviation in the physiological animation, that is, the feedback lags behind the student's operation.

[0098] If the synchronization deviation index is lower than the synchronization deviation lower threshold, it means that the physiological animation has an advance deviation, and the feedback is over-synchronized or ahead of the student's operation.

[0099] According to the deviation type, the response time of the virtual feedback is appropriately adjusted to reduce lag or introduce delay appropriately:

[0100] When a hysteresis deviation is detected in the virtual feedback, the system corrects the hysteresis by reducing the feedback response time. The feedback response time parameters are adjusted to shorten the feedback lag time between the physiological animation and the trainee's operation data to within the preset range.

[0101] When the deviation type is advance, the system introduces a moderate delay so that the physiological animation responds slightly to the operation, thereby avoiding over-synchronization. By increasing the feedback response time, a reasonable deviation from the trainee's operation is achieved.

[0102] Adjust the update frequency of virtual feedback to make the physiological animation's response to the trainee's operation more in line with the actual synchronization requirements:

[0103] In cases where the deviation is large, the system will appropriately adjust the update frequency of the virtual feedback based on the frequency of the trainee's operation to ensure that the physiological animation follows the trainee's operation changes more accurately.

[0104] A smoothing algorithm is used to adjust the update frequency so that the response frequency of the virtual feedback gradually transitions to a range that meets actual operation requirements, avoiding animation jitter or instability caused by frequent switching of frequencies.

[0105] During the process of adjusting the update frequency, the system monitors the effect of the frequency adjustment in real time to ensure that the physiological animation response frequency is consistent with the update frequency of the trainee's operation data.

[0106] During the correction process, real-time monitoring of synchronization is performed, and response parameters are adjusted dynamically to maintain synchronization within a reasonable range:

[0107] During the entire correction process, the system continuously monitors the synchronization status of physiological animation and physical feedback, and dynamically determines the changing trend of synchronization deviation by analyzing real-time data. According to the deviation data monitored in real time, the system adaptively adjusts the response time and update frequency parameters to ensure that the physiological animation and the trainee's operation maintain reasonable synchronization during the training process. If the synchronization is detected to deviate from the preset range again, the system will perform the above adjustment process again until the synchronization is maintained within a reasonable range.

[0108] By combining optical flow analysis with deep convolutional generative adversarial networks, the 3D rendering layer information is analyzed to evaluate the visual penetration of objects in each layer. Specifically:

[0109] Get the current 3D rendering layer information, including the spatial position, texture characteristics and structural relationships of all objects in the scene:

[0110] Among them, spatial position refers to the specific coordinates of an object in three-dimensional space, texture features describe the visual details of the object's surface, and hierarchical relationships define the hierarchical order of each object in the scene.

[0111] The 3D rendering layer information can be represented by a 3D rendering layer information data matrix, where each row represents the information of an object, including coordinates, texture features, and hierarchical relationships. The 3D rendering layer information data matrix is ​​represented as: Among them, M is the 3D rendering layer information data matrix, X j , Y j , Z j represents the three-dimensional space coordinates of the jth object, T j represents the texture feature of the jth object; L j represents the hierarchical relationship of the jth object; m represents the total number of all objects in the scene, j represents the number of the object in the scene, j = 1, 2, ..., m.

[0112] The optical flow algorithm is used to process the 3D rendering layer information of continuous frames to generate the optical flow field features of each layer of objects in the time series:

[0113] The optical flow algorithm calculates the motion changes of each object in the time series and generates a motion vector field, which is expressed as: F = {(u j , v j ,w j )|j=1,2,...,m}; where F is the motion vector field, u j , v j , w j Respectively represent the motion components of the jth object in the X, Y, and Z directions (three-dimensional coordinates);

[0114] Based on the optical flow field, an optical flow feature matrix is ​​constructed to describe the motion characteristics of objects in each layer on the time axis: Among them, G is the optical flow feature matrix.

[0115] The optical flow feature matrix is ​​the input of the subsequent deep convolution generation adversarial network, which contains the motion change information of the object in consecutive frames.

[0116] Based on the extracted optical flow field features, a deep convolutional generative adversarial network model is constructed to learn the visual features and hierarchical relationships of each layer of rendering information in the three-dimensional scene:

[0117] The deep convolutional generative adversarial network model consists of a generator and a discriminator. The generator network receives the optical flow feature matrix as input and generates new rendering data; the discriminator network determines the difference between the generated data and the real data.

[0118] The generator network takes the optical flow feature matrix as input and generates an output matrix that simulates 3D rendering, which is expressed as: in, is the output matrix of simulated 3D rendering, f gen is the mapping function of the generator network.

[0119] The generated output matrix of simulated three-dimensional rendering is as close as possible to the real three-dimensional rendering layer information data matrix.

[0120] The discriminator optimizes the generator by judging the difference between the output matrix of the simulated 3D rendering and the 3D rendering layer information data matrix, which is expressed as a loss function: Among them, L disc is the loss function, L disc Adversarial training is used to make the distribution of generated data closer to that of real data; D(·) is the probability value function output by the discriminator.

[0121] Based on the deep convolutional generative adversarial network model, the penetration analysis of the 3D rendering layer information is performed to evaluate the visual penetration of objects in each layer:

[0122] The trained deep convolutional generative adversarial network model is used to analyze the 3D rendering layer information and extract the penetration features of objects in each layer. The penetration features reflect whether the object's hierarchical position is correctly displayed during the rendering process to avoid confusion between layers.

[0123] The visual penetration coefficient is calculated based on the penetration characteristics to quantify the visual penetration degree of the rendering layer. The calculation formula of the visual penetration coefficient is: Among them, P represents the visual penetration coefficient, n is the total number of rendering layers, and d k The k-th layer rendering depth value extracted by the deep convolutional generative adversarial network, r k is the true depth value of the object in the kth layer, where k is the number of the rendering layer.

[0124] The visual penetration coefficient is used to measure the degree of realistic display of the rendering level and to determine whether there is a visual penetration problem.

[0125] The larger the visual penetration coefficient, the higher the visual penetration of objects in each layer. This usually means that the hierarchy between the rendered layers is not clear enough, which is more likely to cause visual confusion between the layers, making it impossible for students to accurately judge the positional relationship between the layers.

[0126] The sparse coding network is used to extract the features of the pathological feature images of each viewpoint, and the gradient adaptive contrast algorithm is used to evaluate the severity of the visual deviation between viewpoints. Specifically:

[0127] Acquire images of the same pathological scene from different perspectives and calibrate the imaging parameters of each image:

[0128] The same pathological scene is imaged from different perspectives to ensure that the same pathological features are included in multiple perspectives. The images of each perspective are collected by a preset camera system to provide high resolution and visual details that truly reflect the pathological features.

[0129] After acquisition, the imaging parameters of each image are calibrated to ensure the consistency of image quality between viewing angles. Imaging parameters include light intensity, angle, distance, etc. The calibration process mainly adjusts these parameters through image processing algorithms to ensure that the brightness, contrast and detail clarity of all images are displayed under the same standards, reducing the differences in non-pathological features introduced by changes in viewing angles.

[0130] Each image is stored as a feature image matrix, each matrix contains the pixel distribution and image attribute information of the pathological image under the viewing angle. All viewing angle images form a multi-view pathological image set, where the total number of viewing angles is regarded as a fixed value.

[0131] The images of each view are processed through a sparse coding network, the key pathological feature information of each view image is extracted and encoded, and an independent feature map representation is generated:

[0132] The image matrices of each view are input into the sparse coding network. Each image is first preprocessed to extract the main feature areas in the image, ensuring that the network focuses on the pathological feature areas and ignores non-important background information.

[0133] The sparse coding network extracts features from the input image matrix through multi-layer convolution and generates the corresponding feature map. The output of the sparse coding network is set as the key pathological features of each view image. The dimension of the feature vector is a predetermined value, where each component represents an attribute of the pathological feature.

[0134] The feature mapping matrix finally output by the sparse coding network includes the image feature maps of all perspectives, which is used for the subsequent analysis of the feature correspondence between perspectives.

[0135] The feature maps of images from different viewpoints are input into the feature association model, the corresponding relationship of pathological features in images from each viewpoint is analyzed, and a feature comparison vector across viewpoints is generated:

[0136] The feature maps generated by the sparse coding network are gradually input into the feature association model. The feature association model is responsible for establishing the feature relationship of different perspectives and ensuring that the association between the features of each perspective is consistent with the consistency of the pathological features.

[0137] The association model generates a feature contrast vector across views based on the input feature map. The feature contrast vector is used to represent the correspondence between the pathological features of the two views and is defined as a feature difference vector, which is used to represent the feature difference between the two views.

[0138] The feature contrast vectors between all views are combined into a feature contrast vector matrix to represent the feature correspondence matrix across views. The feature contrast vector will be used for subsequent gradient adaptive contrast analysis.

[0139] The feature contrast vector matrix can be expressed as: Among them, V is the feature contrast vector matrix and v is the total number of viewing angles.

[0140] For example, V 12 Represents the feature contrast vector between view 1 and view 2.

[0141] Based on the feature contrast vector, the gradient adaptive contrast algorithm is applied to gradually calculate the difference amplitude of pathological features between different perspectives:

[0142] Based on the feature contrast vector matrix, the gradient adaptive contrast algorithm is applied to calculate the feature difference of each pair of perspectives, and the feature difference amplitude index is calculated, which is expressed as: Among them, A qp is the feature difference amplitude index, which indicates the feature difference amplitude between viewing angle q and viewing angle p; F q，r and F p，r F q and F p The rth component of ; c represents the dimension of the eigenvector; r represents the number of the component; ||V qp ||Euclidean distance representing feature differences; F q represents the feature vector of the qth perspective, which contains the pathological feature information under this perspective; p The feature vector representing the p-th perspective contains the pathological feature information under another perspective; q represents the number of the first perspective, p represents the number of the second perspective, and the perspectives q and p are used to distinguish the pathological feature images under different perspectives.

[0143] The difference in pathological features between different viewing angles was summarized and calculated to quantify the severity of visual deviation between viewing angles:

[0144] The feature difference amplitude index is summed and the average value is calculated as the visual deviation index, which is expressed as: Among them, S y represents the visual deviation index, Indicates the number of combinations between viewing angles (i.e. the total number of viewing angle pairs).

[0145] The severity of the visual deviation between perspectives is quantified based on the visual deviation index. Specifically, the larger the visual deviation index, the higher the severity of the visual deviation between perspectives. This means that when observing the same pathological feature from different perspectives, the feature differences between images are large, the consistency between perspectives is low, and the presentation of pathological features at different perspectives is inconsistent, resulting in trainees or users being unable to clearly and accurately identify the same pathological feature when switching perspectives, reducing the credibility of multi-perspective display. Severe visual deviation will make it difficult for trainees to understand the real spatial relationship of pathological features during training, affecting the accuracy of judgment. In addition, excessive deviation may also cause visual dislocation or discomfort when switching perspectives, interfering with trainees' learning effect and true perception of pathological features.

[0146] Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviation between viewing angles, it is determined whether the accuracy risk of virtual pathology display is within the allowable range, specifically:

[0147] Set the threshold of the visual penetration coefficient, denoted as α. α is used to evaluate whether the visual penetration of objects in each layer is within a reasonable range, ensuring that the layering is clear and there is no confusion caused by excessive penetration.

[0148] The threshold of the visual deviation index is set, denoted as β. β is used to evaluate whether the severity of the visual deviation between viewing angles is within an acceptable range to ensure the consistency of pathological features when observing from multiple viewing angles.

[0149] Compare the visual penetration coefficient with α: if the visual penetration coefficient is less than or equal to α, it is judged that the visual penetration of objects in each layer is within a reasonable range; if the visual penetration coefficient is greater than α, it is judged that there is a high risk of layer confusion.

[0150] Compare the visual deviation index with β: if the visual deviation index is less than or equal to β, it is judged that the visual deviation between viewing angles is within the allowable range; if the visual deviation index is greater than β, it is judged that the consistency between viewing angles is poor, which may affect the accuracy of pathological display.

[0151] When the visual penetration of objects in each layer is within a reasonable range and the visual deviation between viewing angles is within an allowable range, the accuracy risk of the virtual pathology display is determined to be within an allowable range; otherwise, the accuracy risk of the virtual pathology display is determined to be not within the allowable range.

[0152] When the accuracy risk of virtual pathology display is within the allowable range, it means that the accuracy risk of using model animation to display the pathological state of virtual patients is low. In this case, the display of the pathological state of virtual patients is visually closer to the real situation, which helps trainees accurately identify and understand the spatial relationship and depth details of pathological features from different perspectives.

[0153] The threshold of the visual penetration coefficient is used to detect whether the visual penetration of objects in each layer is within a reasonable range, to ensure that the hierarchical relationship is clear and to avoid confusion caused by excessive penetration. The threshold of the visual deviation index is used to evaluate the consistency between perspectives to ensure that the pathological features remain consistent when observed from different perspectives. By comparing the actual visual penetration coefficient and visual deviation index with their respective thresholds, it is possible to effectively determine whether the accuracy risk of the virtual display is within an acceptable range. If both the penetration and perspective consistency meet the threshold requirements, it means that the display effect is real and reliable, and the risk is controllable; otherwise, it is necessary to further adjust the rendering or correct the perspective consistency to ensure that students can accurately understand the spatial position and depth relationship of the pathological features.

[0154] Example 2

[0155] The difference between Example 2 of the present invention and Example 1 is that this example introduces a comprehensive scenario training system for cardiopulmonary resuscitation.

[0156] Figure 2A structural schematic diagram of a comprehensive scenario training system for cardiopulmonary resuscitation of the present invention is given, a comprehensive scenario training system for cardiopulmonary resuscitation, including a synchronization deviation evaluation module, a synchronization anomaly detection module, a synchronization deviation correction module, a visual penetration analysis module, a visual deviation evaluation module and a display accuracy judgment module.

[0157] Synchronization deviation evaluation module: The student operation data and virtual feedback data are analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback.

[0158] Synchronization anomaly detection module: analyzes the synchronization deviation rate between physiological animation and physical feedback within a preset interval to determine whether the synchronization between physiological animation and physical feedback is abnormal.

[0159] Synchronization deviation correction module: When the synchronization between physiological animation and physical feedback is abnormal, the synchronization deviation is corrected.

[0160] Visual penetration analysis module: When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer.

[0161] Visual deviation assessment module: When the synchronization between physiological animation and physical feedback is normal, the pathological feature images of each perspective are extracted through a sparse coding network, and the severity of the visual deviation between perspectives is evaluated using a gradient adaptive contrast algorithm.

[0162] Display accuracy judgment module: Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it determines whether the accuracy risk of the virtual pathology display is within the allowable range.

[0163] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0164] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0165] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0168] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0170] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0171] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0172] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A comprehensive scenario training method for cardiopulmonary resuscitation, characterized in that: The steps include: The students’ operation data and virtual feedback data were analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback. Analyze the synchronization deviation rate between the physiological animation and the physical feedback within a preset interval to determine whether the synchronization between the physiological animation and the physical feedback is abnormal; When the synchronization between physiological animation and physical feedback is abnormal, correct the synchronization deviation; When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer; The pathological feature images of each view are extracted through a sparse coding network, and the severity of visual deviation between viewpoints is evaluated using a gradient adaptive contrast algorithm. Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it is determined whether the accuracy risk of the virtual pathology display is within the allowable range.

2. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 1, characterized in that: The students’ operation data and virtual feedback data were analyzed through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback. Specifically: Acquire the trainee operation data and corresponding virtual feedback data during the cardiopulmonary resuscitation training, and pre-process the trainee operation data and virtual feedback data; Based on the pre-processed student operation data and virtual feedback data, a hidden variable Markov model is constructed; Clarify the state space, observation sequence, and transition probability matrix of the latent variable Markov model; The expectation maximization algorithm is used to train the parameters of the hidden variable Markov model, and the optimal parameters of the hidden variable Markov model are solved through iterative optimization; Using the trained hidden variable Markov model, the trainees' operation data and virtual feedback data were sequence analyzed to calculate the synchronization deviation rate between physiological animation and physical feedback.

3. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 2, characterized in that: The synchronization deviation rate between the physiological animation and the physical feedback within the preset interval is analyzed to determine whether the synchronization between the physiological animation and the physical feedback is abnormal, specifically: Obtaining an average value of the overall synchronization deviation rate within a preset interval, and marking the average value of the overall synchronization deviation rate within the preset interval as a synchronization deviation index; Set the upper and lower thresholds of synchronization deviation; When the synchronization deviation index is between the synchronization deviation upper limit threshold and the synchronization deviation lower limit threshold, it is determined that the synchronization between the physiological animation and the physical feedback is normal; otherwise, it is determined that the synchronization between the physiological animation and the physical feedback is abnormal.

4. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 3, characterized in that: When the synchronization between physiological animation and physical feedback is abnormal, correct the synchronization deviation, specifically: Analyze the synchronization deviation between physiological animation and physical feedback, and determine the direction of the deviation; The response time of the virtual feedback is appropriately adjusted according to the deviation type; Adjust the update frequency of virtual feedback; Monitor synchronization in real time during the correction process and dynamically adjust response parameters.

5. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 4, characterized in that: By combining optical flow analysis with deep convolutional generative adversarial networks, the 3D rendering layer information is analyzed to evaluate the visual penetration of objects in each layer. Specifically: Get the current 3D rendering layer information, including the spatial position, texture features and structural relationships of all objects in the scene; The optical flow algorithm is used to process the 3D rendering layer information of continuous frames to generate the optical flow field features of each layer of objects in the time series; A deep convolutional generative adversarial network model is constructed based on the extracted optical flow field features to learn the visual features and hierarchical relationships of each layer of rendering information in the three-dimensional scene; Based on the deep convolutional generative adversarial network model, the penetration analysis of the 3D rendering layer information is performed to evaluate the visual penetration of objects in each layer.

6. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 5, characterized in that: Based on the deep convolutional generative adversarial network model, the penetration analysis of the 3D rendering layer information is performed to evaluate the visual penetration of objects in each layer. Specifically: Calculate the visual penetration coefficient, the calculation formula is: Among them, P represents the visual penetration coefficient, n is the total number of rendering layers, and d k The k-th layer rendering depth value extracted by the deep convolutional generative adversarial network, r k is the true depth value of the object in the kth layer, where k is the number of the rendering layer.

7. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 6, characterized in that: The sparse coding network is used to extract the features of the pathological feature images of each viewpoint, and the gradient adaptive contrast algorithm is used to evaluate the severity of the visual deviation between viewpoints. Specifically: Acquire images of the same pathological scene from different perspectives and calibrate the imaging parameters of each image; The images of each view are processed through a sparse coding network to extract and encode the key pathological feature information of each view image and generate an independent feature map representation; The feature maps of images from different viewpoints are input into the feature association model, the corresponding relationship of the pathological features in the images from each viewpoint is analyzed, and a feature comparison vector across viewpoints is generated; Based on the feature contrast vector, a gradient adaptive contrast algorithm is applied to gradually calculate the difference amplitude of pathological features between different viewing angles; The difference in pathological features between different viewing angles was summarized and calculated to quantify the severity of visual deviation between viewing angles.

8. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 7, characterized in that: Based on the feature contrast vector, the gradient adaptive contrast algorithm is applied to gradually calculate the difference amplitude of pathological features between different perspectives, specifically: The gradient adaptive contrast algorithm is applied to calculate the feature difference of each pair of perspectives, and the feature difference amplitude index is calculated, which is expressed as: Among them, A qp is the feature difference magnitude index, F q，r and F p，r F q and F p The rth component of , c represents the dimension of the eigenvector, r represents the number of the component, ||V qp ||Euclidean distance representing feature differences, F q represents the feature vector of the qth view, F p represents the feature vector of the pth view, q represents the number of the first view, and p represents the number of the second view.

9. A comprehensive scenario training method for cardiopulmonary resuscitation according to claim 8, characterized in that: Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviation between viewing angles, it is determined whether the accuracy risk of virtual pathology display is within the allowable range, specifically: Set the threshold of the visual penetration coefficient, denoted as α; set the threshold of the visual deviation index, denoted as β; If the visual penetration coefficient is less than or equal to α, it is judged that the visual penetration of objects in each layer is within a reasonable range; if the visual penetration coefficient is greater than α, it is judged that there is a high risk of layer confusion; If the visual deviation index is less than or equal to β, it is determined that the visual deviation between viewing angles is within the allowable range; If the visual deviation index is greater than β, it is judged that the consistency between perspectives is poor; When the visual penetration of objects in each layer is within a reasonable range and the visual deviation between viewing angles is within an allowable range, the accuracy risk of the virtual pathology display is determined to be within an allowable range.

10. A comprehensive scenario training system for cardiopulmonary resuscitation, used to implement a comprehensive scenario training method for cardiopulmonary resuscitation according to any one of claims 1 to 9, characterized in that: It includes a synchronization deviation assessment module, a synchronization anomaly detection module, a synchronization deviation correction module, a visual penetration analysis module, a visual deviation assessment module, and a display accuracy judgment module; Synchronization deviation evaluation module: Analyze the trainee operation data and virtual feedback data through the hidden variable Markov model to evaluate the synchronization deviation rate between physiological animation and physical feedback; Synchronization anomaly detection module: analyzes the synchronization deviation rate between physiological animation and physical feedback within a preset interval to determine whether the synchronization between physiological animation and physical feedback is abnormal; Synchronization deviation correction module: when the synchronization between physiological animation and physical feedback is abnormal, the synchronization deviation is corrected; Visual penetration analysis module: When the synchronization between physiological animation and physical feedback is normal, the 3D rendering layer information is analyzed through optical flow analysis combined with deep convolutional generative adversarial networks to evaluate the visual penetration of objects in each layer; Visual deviation assessment module: When the synchronization between physiological animation and physical feedback is normal, the pathological feature images of each view are extracted through a sparse coding network, and the severity of visual deviation between viewpoints is assessed using a gradient adaptive contrast algorithm; Display accuracy judgment module: Based on a comprehensive analysis of the visual penetration of objects in each layer and the severity of visual deviations between viewing angles, it determines whether the accuracy risk of the virtual pathology display is within the allowable range.

Images