Virtual teaching system for bronchoscopy

Abstract

The application relates to the technical field of medical simulation teaching, in particular to a virtual teaching system of a bronchoscope. An acquisition module first acquires the multi-modal operation data of a target operator on a virtual bronchoscope collected in real time through a VR device. An evaluation module generates a capability profile and determines a weak point in capability in combination with the multi-level branch anatomical features and respiratory physiological features of a bronchus. An adjustment module selects training content matched with the capability profile and the weak point in capability from a training library. An interaction module visualizes the operation suggestions of a teacher and superimposes the operation suggestions on the display interface of the VR device through an AR device to dynamically update the display parameters. The application provides targeted teaching scenes for target operators with different degrees of proficiency in combination with the VR device and the AR device, improves the training efficiency and clinical conversion rate, and reduces the understanding difficulty of the target operator based on the visual guidance of the teacher, thereby improving the teaching efficiency of bronchoscope operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical simulation teaching, in particular to a virtual teaching system of a bronchoscope. BACKGROUND

[0002] Bronchoscopy technology is one of the core means for diagnosis and treatment of lung tumors, infectious diseases, diffuse lung diseases, etc. It realizes the visualization observation and accurate operation of each branch of bronchial tree by inserting a slender bronchoscope into the lung through the airway. Because the bronchial tree has a complex multi-level branch structure and presents dynamic deformation with the respiratory cycle, the requirement for bronchoscope operation is high. The traditional bronchoscope teaching is usually based on static airway models for operation teaching, and the teaching process is easy to be out of touch with the actual operation. And the teaching evaluation based on general indicators is easy to lead to the situation that the novice students may be frustrated by too high difficulty, and the skilled persons may be inefficient by too low difficulty. Therefore, the general teaching cannot adapt to individualized training, and the single machine operation is different from the actual collaborative operation scene, resulting in low teaching efficiency of the bronchoscope. SUMMARY

[0003] The purpose of the present application is to provide a virtual teaching system of a bronchoscope to solve the problem of low teaching efficiency of the bronchoscope.

[0004] In order to achieve the above purpose, the present application provides a virtual teaching system of a bronchoscope, which communicates with a VR device and an AR device, and comprises:

[0005] An acquisition module is configured to acquire, by the VR device, multi-modal operation data of a target operator on a virtual bronchoscope collected in real time;

[0006] An evaluation module is configured to generate an ability portrait of the target operator and determine weak points of the target operator based on the multi-modal operation data, in combination with multi-level branch anatomical features and respiratory physiological features of bronchi;

[0007] An adjustment module is configured to select training content matched with the ability portrait and the weak points from a training library according to the ability portrait and the weak points, wherein the training content comprises single-role hierarchical training, multi-role collaborative training and weak point reinforcement training;

[0008] An interaction module is configured to acquire operation suggestions marked by a teacher through the AR device in a training process, adjust display parameters of the VR device based on the operation suggestions, and update a display interface of the VR device based on the display parameters.

[0009] The present application has the following beneficial effects:

[0010] The application provides a virtual teaching system of a bronchoscope in communication with a VR device and an AR device respectively. A obtaining module first obtains multi-modal operation data of a target operator on a virtual bronchoscope collected in real time through the VR device. An evaluation module then generates a capability profile of the target operator and determines weak points in capability based on the multi-modal operation data, in combination with multi-level branch anatomical features and respiratory physiological features of the bronchus. Evaluating the operation capability of the target operator based on the multi-modal operation data can improve the accuracy of the evaluation, solve the problem of fuzzy capability positioning, and provide a high-quality data basis for the selection of subsequent training scenarios.

[0011] Next, an adjusting module selects training content matching the capability profile and the weak points in capability from a training library according to the capability profile and the weak points in capability. The training content can include single-role hierarchical training, multi-role collaborative training, and weak point reinforcement training. The single-role hierarchical training, in combination with weak point reinforcement, can develop individualized training content for target operators of different levels, reducing frustration caused by excessive difficulty or inefficient repetition of mastered content. The multi-role collaborative training can fully replicate the clinical operation process, enabling the target operator to form a clinically standardized muscle memory and a sense of cooperation in the virtual environment, reducing the clinical conversion period and the problem of disconnection between training and clinical practice.

[0012] An interaction module visualizes and superimposes the operation suggestions of the teacher on the display interface of the VR device through the AR device to dynamically update the display parameters. Compared with verbal guidance and abstract two-dimensional diagrams, this can enable the target operator to understand the correlation between operation defects and anatomy in real time, improving the efficiency of teaching feedback.

[0013] The application combines the VR device and the AR device, accurately evaluates the target operator, develops individualized hierarchical training scenarios, weak point training scenarios, and collaborative training scenarios, provides targeted teaching scenarios for target operators of different levels, reduces the clinical conversion period, and improves training efficiency and clinical conversion rate. Moreover, based on the visual guidance of the teacher in three-dimensional space, the understanding difficulty of the target operator is reduced, thereby improving the teaching efficiency of bronchoscope operation.

[0014] Other features and advantages of the application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 An application scenario diagram of the virtual teaching system of the bronchoscope in the embodiments of the application;

[0016] Figure 2 A structure diagram of the virtual teaching system of the bronchoscope provided in an embodiment of the application;

[0017] Figure 3 a first training scenario provided in an embodiment of the present application;

[0018] Figure 4 a second training scenario provided in an embodiment of the present application;

[0019] Figure 5 a third training scenario provided in an embodiment of the present application;

[0020] Figure 6 a structure schematic diagram of a virtual teaching system of a bronchoscope in another embodiment of the present application.

[0021] Legend of reference signs

[0022] 1, controller; 2, virtual bronchoscope; 3, VR device; 4, AR device; 200, virtual teaching system of bronchoscope; 201, acquisition module; 202, evaluation module; 203, adjustment module; 204, interaction module; 205, alarm module; 206, traceability module. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0024] In the description of the present application, it should be understood that the terms “first”, “second” are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first”, “second” can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of “multiple” is two or more, unless otherwise specifically limited. In the present application, the word “exemplary” is used to mean “serving as an example, instance, or illustration”. Any embodiment described as “exemplary” in the present application is not necessarily to be construed as preferred or advantageous over other embodiments. In order to enable any person skilled in the art to implement and use the present application, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that those skilled in the art can realize the present application without using these specific details. In other instances, well-known structures and processes will not be described in detail in order to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the shown embodiments, but is consistent with the broadest scope of principles and features disclosed in the present application.

[0025] Embodiments of the present application introduce virtual reality (VR) and augmented reality (AR) technologies in a virtual bronchoscopy teaching system. An immersive virtual scene is constructed through VR technology, accurately restoring the anatomical features of multi-level bronchial branches and respiratory deformation. Combined with AR technology, visualization guidance is achieved, realizing the combination of immersive training and real-time guidance. In this way, personalized evaluation of the operator, targeted scene training, and intuitive feedback can be achieved, improving teaching quality and clinical conversion rate.

[0026] Figure 1 An application scenario diagram of the virtual bronchoscopy teaching system in embodiments of the present application is shown. The application scenario can include a controller 1, a virtual bronchoscope 2, a VR device 3, and an AR device 4. The controller 1 communicates with the virtual bronchoscope 2, the VR device 3, and the AR device 4, respectively.

[0027] The controller 1 integrates the virtual bronchoscopy teaching system, which can include a memory and a processor. The memory is configured to store instructions and data, and the processor is configured to call the instructions and data from the memory and execute the steps of the virtual bronchoscopy teaching system when running the instructions. In one example, the controller 1 can receive operation data related to the virtual bronchoscope 2 collected by the VR device 3, such as the mirror body advancement trajectory and force, from the target operator. The target operator is the person who operates the virtual bronchoscope 2, such as a trainee. The controller 1 can also receive operation suggestions of the teacher obtained by the AR device 4, such as annotation information and voice instructions, etc. For example, the controller 1 can also execute training scene switching instructions, force feedback threshold adjustment instructions, etc., and send the instructions to the VR device.

[0028] The virtual bronchoscope 2 is a digital twin instrument generated by software modeling of the virtual teaching system. The morphology and operation characteristics of the virtual bronchoscope 2 match the clinical real bronchoscope, there is no physical hardware, and it only exists in the virtual scene. The controller 1 sends the operation instructions of the target operator, performs navigation, lesion treatment, etc. in the scene of the virtual bronchoscope 2 constructed by the VR device, and then generates operation state data in real time, which is sent to the VR device 3 for display on the display device of the VR device 3.

[0029] The VR device 3 is a hardware complex that provides an immersive virtual environment, which can include a display interface of the VR device, a force feedback handle, a position tracking sensor, and the like, and is an entry for a user to interact with a scene of the virtual bronchoscope 2. The VR device 3 can display a scene matched with the virtual bronchoscope 2 based on selected training content, and update an operation view and parameter data of the virtual bronchoscope 2 in real time. During the operation of the virtual bronchoscope 2, the force feedback handle can be used to collect the force of the user controlling the virtual bronchoscope 2, and the position sensor can be used to collect operation data such as a mirror body advancing track and a steering speed, as basic data for capability assessment. In the case of a risk in the operation, visual and tactile feedback can be provided through the display interface and the force feedback handle to timely prompt the target operator of the safety risk.

[0030] The AR device 4 is an augmented reality hardware that realizes fusion of a virtual scene and real-time guidance, which can be an intelligent reality terminal that superimposes virtual information on a real environment in real time through optical see-through technology, retains intuitive perception of a real scene, and supplements invisible information in the visual field environment through computer-generated virtual content to realize interaction between the displayed world and digital information. As an example, the AR device 4 can be a head-mounted AR device. The AR device can include an interactive interface, a gesture recognition sensor, a voice sensor, and the like to capture the actions of a teacher. For example, in the scene of the virtual bronchoscope operation in the embodiment of the present application, the teacher can mark the operation suggestion through the AR device 4. The AR device 4 confirms the gesture drawing operation of the teacher through the gesture recognition sensor, or confirms the voice suggestion of the teacher based on the voice sensor, and then converts the gesture feedback operation or the voice suggestion into marking information. Alternatively, the marking information is confirmed through a virtual marking tool of the interactive interface. Based on the marking information, the operation suggestion of the teacher can be obtained, and the operation suggestion is sent to the controller 1 in real time, and the controller 1 synchronizes the operation suggestion to the VR device 3 to superimpose the marking information on the display interface of the VR device 3.

[0031] In the embodiment of the present application, a teaching process can be formed in which the target operator controls the virtual bronchoscope 2, the VR device 3 collects data and displays a scene, the controller 1 adjusts the training content after evaluating the target operator, and the teacher can superimpose the operation suggestion to the display interface of the VR device 3 through the AR device 4. In this way, the clinical characteristics of the bronchoscope operation are restored, and the scene single, feedback after, system difficulty, and a large number of instruments required in the traditional teaching are solved through virtual fusion technology, which can improve the teaching efficiency of the bronchoscope.

[0032] It can be understood that, Figure 1The electronic devices in the application scenarios of the bronchoscope virtual teaching system shown do not constitute a limitation on the embodiments of the present application, that is, the number and types of devices included in the application scenarios of the bronchoscope virtual teaching system, or the number and types of devices included in each electronic device do not affect the overall implementation of the technical solutions in the embodiments of the present application, and can all be counted as equivalent replacements or derivatives of the technical solutions claimed in the embodiments of the present application.

[0033] The controller 1 in the embodiments of the present application can be a standalone device, or a device network or device cluster composed of devices. For example, the controller 1 described in the embodiments of the present application includes but is not limited to a computer, a network host, a single network device, a plurality of network device sets, or a cloud device composed of a plurality of devices. The cloud device is composed of a large number of computers or network devices based on cloud computing.

[0034] Those skilled in the art can understand that Figure 1 The application scenarios shown in the embodiments of the present application are only one of the application scenarios corresponding to the technical solutions of the present application, and do not constitute a limitation on the application scenarios of the technical solutions of the present application. Other application scenarios can include more or fewer electronic devices than Figure 1 The application scenarios shown in the embodiments of the present application are only one of the application scenarios corresponding to the technical solutions of the present application, and do not constitute a limitation on the application scenarios of the technical solutions of the present application. Other application scenarios can include more or fewer electronic devices than Figure 1 In the embodiments of the present application, only one electronic device is shown. It can be understood that the application scenario of the bronchoscope virtual teaching system can also include one or more other electronic devices, and the specific number is not limited here.

[0035] It should be noted that Figure 1 The application scenario of the bronchoscope virtual teaching system shown is only an example. The application scenario of the bronchoscope virtual teaching system described in the embodiments of the present application is to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0036] Based on the above-mentioned application scenario of the bronchoscope virtual teaching system, embodiments of the bronchoscope virtual teaching system are proposed. The modules and units in the embodiments of the present application can communicate with each other. The following will be described in detail with reference to the accompanying drawings.

[0037] Figure 2 A structural schematic diagram of a bronchoscope virtual teaching system 200 provided in an embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the bronchoscope virtual teaching system 200 can be integrated in a controller 1, and communicates with a VR device 3 and an AR device 4, and can include an acquisition module 201, an evaluation module 202, an adjustment module 203, and an interaction module 204. Figure 2

[0038] ​The acquisition module 201 is configured to acquire, by the VR device 3, multi-modal operation data of the target operator on the virtual bronchoscope 2 in real time.

[0039] The multi-modal operation data refers to data generated by the target operator when operating the virtual bronchoscope and covering multi-dimensional operation of the bronchoscope. For example, the multi-modal operation data can include anatomical recognition data, breathing operation parameters, and lesion treatment parameters, etc. In an example, the target operator can first operate the virtual bronchoscope 2 through the force feedback handle of the VR device 3. Then, the operation data is acquired in different dimensions to obtain the multi-modal operation data. The multi-modal operation data is preprocessed to align the time stamp of the original data and filter the abnormal values, so as to generate a standardized data set and transmit it to the evaluation module 202. Compared with general data, evaluating the operation ability of the target operator based on the multi-modal operation data can improve the accuracy of the evaluation, solve the problem of fuzzy ability positioning, and provide a high-quality data basis for the selection of subsequent training scenarios.

[0040] The evaluation module 202 is configured to generate an ability portrait of the target operator based on the multi-modal operation data, in combination with the multi-level branch anatomical features and respiratory physiological features of the bronchus, and determine the weak points of the target operator.

[0041] The ability portrait is generated based on the multi-modal data and reflects the quantitative results of the multi-dimensional ability of the target operator, which can include a comprehensive ability score and a corresponding grade classification corresponding to the comprehensive ability score. The ability breathing weak point is an operation item related to high-frequency errors in bronchoscope operation, which can represent the specific shortcomings of the target operator. In an example, index data can be obtained from multiple dimensions, such as the accuracy of recognizing key branches of the bronchus, the breathing adaptation degree, and the treatment accuracy of small bronchus lesions, etc. Based on the obtained index data, normalization and weighted fusion can be performed to obtain a comprehensive ability score, thereby determining the ability portrait of the target operator. In another example, a decision tree corresponding to the bronchoscope can be constructed, the multi-modal data can be input, and layer-by-layer judgment can be performed, thereby determining the weak points of the target operator. Compared with traditional general evaluation, the multi-level branch anatomical features and respiratory physiological features of the bronchus can be fused to more accurately classify the ability and locate the specific weak points.

[0042] The adjustment module 203 is configured to select training content matching the ability portrait and the weak points from the training library according to the ability portrait and the weak points.

[0043] The training content and ability portrait of the embodiments of the present application are precisely matched with weak points, and can include single-role hierarchical training, multi-role collaborative training, and weak point reinforcement training. The single-role hierarchical training is a single-person training scene that is matched according to the ability level and has progressive difficulty. The multi-role collaborative training is a multi-person collaborative scene that simulates the division of labor among the main surgeon, the assistant, and the instrument nurse in the clinic. The weak point reinforcement training is a special scene generated for the short board of positioning. By precisely selecting the training level and reinforcing the weak points, a precise training plan can be developed for each target operator, reducing the frustration of practicing high difficulty at the novice level and the low efficiency of practicing low level at the expert level, and improving training efficiency. Moreover, through collaborative training, the clinical process can be replicated, the conversion period from training to clinic can be reduced, and the clinical conversion capability can be improved.

[0044] The interaction module 204 is configured to obtain the operation suggestion marked by the teacher through the AR device 4 during the training process, adjust the display parameters of the VR device 3 based on the operation suggestion, and update the display interface of the VR device 3 based on the display parameters.

[0045] The operation suggestion is guidance information associated with the operation scene of the virtual bronchoscope 2 input by the teacher through the AR device 4. For example, it can include path marking, breathing prompt, and instrument selection suggestion, etc. The display parameter is the attribute corresponding to the display interface of the VR device, which is used to visually present the operation suggestion. For example, the superimposed position of the marking, the color mode of the highlight, and the magnification, etc. In one example, the AR device 4 can receive multi-modal input of the teacher, such as a gesture-drawn navigation path, a voice-input prompt, and marking information determined by a virtual marking tool, etc. After processing by the controller 1, the display parameters of the VR device 3 are adjusted, and the processed display parameters are updated to the display interface of the VR device 3 in real time, so that the target operator can see the superimposed operation suggestion in the virtual scene. In this way, compared with oral guidance and abstract mode of two-dimensional plan, the target operator can understand the association between operation defects and anatomy in real time through visual operation suggestion, and the efficiency of teaching feedback is improved.

[0046] The embodiments of the present application combine the VR device 3 and the AR device 4, precisely evaluate the target operator, develop individualized hierarchical training scenes, weak point training scenes, and collaborative training scenes, can provide targeted teaching scenes for target operators of different levels, and reduce the clinical conversion period, improve the training efficiency and the clinical conversion rate. Moreover, based on the visual guidance of the teacher in the three-dimensional space, the understanding difficulty of the target operator is reduced, thereby improving the teaching efficiency of bronchoscope operation.

[0047] In the embodiments of the present application, the acquisition module 201 can include a first acquisition unit, a second acquisition unit, and a third acquisition unit.

[0048] The first acquisition unit is configured to acquire identification parameters of the target operator in the multi-level branches of the bronchus. The identification parameters of the multi-level branches refer to the identification operation data of the target operator on the clinically high-frequency error branches in the bronchial tree. For example, the identification operation parameters of the carina of the trachea, the bifurcation of the basal segment of the left lower lobe, and the stenosis segment of the middle lobe of the right lung. Therefore, the identification parameters can include the correct number of times of the virtual bronchoscope operation for the carina of the trachea, the bifurcation of the basal segment of the left lower lobe, and the stenosis segment of the middle lobe of the right lung, and the first total number of operations. The correct number of times refers to the number of times of accurately identifying the above branches in the operation. The first total number of operations is the total number of identification attempts for the above branches, which can include correct operations and incorrect operations.

[0049] In one example, when the virtual bronchoscope 2 is advanced to the transition region from the main bronchus to the lobar bronchus, the first acquisition unit can activate and lock the three target branches of the carina of the trachea, the bifurcation of the basal segment of the left lower lobe, and the stenosis segment of the middle lobe of the right lung. Then, data statistics are performed for the three target branches, structured parameters are generated, and transmitted to the evaluation module 202. Compared with the data redundancy caused by covering all branches, focusing only on clinically critical high-frequency error branches can directly reflect the navigation operation ability of the bronchoscope, reduce the amount of calculation, and improve the identification efficiency.

[0050] The second acquisition unit is configured to acquire breathing operation parameters of the target operator in the multi-breathing phase operation. The breathing operation parameters are data reflecting the operation safety and effectiveness of the target operator in the dynamic changes of the breathing cycle. The breathing operation parameters can include the effective operation number and the second total operation number of the bronchus in the inspiration phase, and the airway wall collision number and the third total operation number of the bronchus in the expiration phase. The effective operation number is the number of safe operations in which the virtual bronchoscope 2 is smoothly advanced without collision branch switching in the inspiration phase. The second total operation number is the total number of operation attempts in the inspiration phase. The airway wall collision number is the number of times that the virtual bronchoscope 2 contacts the airway wall in the expiration phase. The third total operation number is the total number of operation attempts in the expiration phase.

[0051] In one example, the real breathing frequency can be simulated by the built-in breathing waveform simulator in the virtual scene, the current phase is marked in real time, and the breathing operation parameters are collected in phases. The inspiration phase can display a green waveform, and the expiration phase can display a blue waveform, so that the operator can intuitively obtain the current breathing phase. Then, the data of the inspiration phase and the expiration phase are collected respectively, the operation behavior of the operator in the expiration phase and the inspiration phase is recorded, and the breathing operation parameters are obtained. Binding the operation data with the dynamic deformation of the bronchus in the breathing, the breathing and operation cooperation capability data can be accurately captured, providing a data basis for subsequent breathing adaptation degree calculation, so that the evaluation can truly predict the risk of airway mucosa damage in the clinic.

[0052] The third acquisition unit is configured to acquire a lesion treatment parameter of the target operator when operating on the lesion. The lesion treatment parameter includes a hemostasis success rate and a residual mass sputum rate for a small bronchus lesion. The lesion treatment parameter is operation data of a core scene of bronchoscope treatment, for example, a treatment parameter for a peripheral small branch lesion. The small bronchus refers to a subsegment bronchus with a diameter less than 3 mm. Due to the limited field of view of the bronchoscope, the instrument is difficult to adapt, and the failure rate of clinical treatment for the small bronchus is also high. The small bronchus lesion refers to a lesion located in the small bronchus region. The evaluation of lesion treatment usually includes the hemostasis success rate and the residual mass sputum rate. The hemostasis success rate is the ratio of the number of cases of controlling bleeding to the total number of cases in the virtual small bronchus bleeding scene. The residual mass sputum rate is the ratio of the number of cases of residual sputum rate after operation to the total number of cases in the virtual sputum accumulation scene. Excessive residual amount can easily block the field of view and affect subsequent operation. Therefore, the hemostasis success rate and the residual mass sputum rate can reflect the lesion treatment level of the target operator when operating on the lesion.

[0053] In one example, when the virtual bronchoscope 2 is advanced to the small bronchus segment, the lesion treatment scene is triggered, and the third acquisition unit starts to acquire data to record the hemostasis success rate and the residual mass sputum rate of the target operator. The hemostasis success rate can be recorded by recording the operation time of the target operator from discovering the bleeding point to stopping bleeding. If the time is less than a set time, for example, 10 s, and the bleeding point is completely closed, it can be determined as a hemostasis success case. The total number of hemostasis cases is obtained by accumulating the treatment attempts of all bleeding scenes in the training period to calculate the hemostasis success rate. The residual mass sputum rate can be measured by a virtual volume sensor. When the residual amount is less than a set percentage of the initial amount, for example, 5%, it can be determined as a qualified removal case. The total number of sputum cases is obtained by accumulating the treatment attempts of all sputum scenes in the training period to calculate the residual mass sputum rate. The hemostasis success rate and the residual mass sputum rate are packaged and transmitted to the evaluation module, which can be used as reference data for the treatment accuracy of the small branch lesion. By focusing on the small bronchus segment of the bronchoscope treatment, the treatment difficulty of the peripheral small bronchus can be solved, and the training effect of the clinical treatment ability can be improved.

[0054] In the embodiments of the present application, the evaluation module 202 can include an index calculation unit, a score determination unit, and a weak point determination unit.

[0055] The index calculation unit is configured to calculate the recognition accuracy, the respiratory adaptation degree, and the fine branch lesion treatment precision based on the multi-modal operation data. The recognition accuracy is the accuracy of the target operator in recognizing the bronchoscopic clinical high-frequency error branch. The respiratory adaptation degree is a comprehensive index combining the effectiveness of operation in the inspiration period and the safety of operation in the expiration period, so as to quantify the synergy of the respiration and the operation, and the value range can be [0, 1], and the higher the value is, the better the synergy is. The fine branch lesion treatment precision is a quality index for the treatment of lesions in the fine branch area. The index is actually in line with the actual characteristics of bronchoscopy, and can focus on the branches prone to navigation errors, the influence of airway dynamic deformation on operation, and the treatment difficulties of peripheral fine bronchus.

[0056] In one example, the index calculation unit can perform the following steps.

[0057] Firstly, the proportion of the correct number of virtual bronchoscopy operations for the tracheal carina, the left lower lobe basal segment bifurcation, and the right middle lobe stenosis segment of the bronchus in the first total operation number in a set time period is calculated to obtain the recognition accuracy. The set time period can be set according to the training period. For example, assuming that the first total operation number is 15 times and the correct number is 12 times, the recognition accuracy can be 80%.

[0058] Secondly, the effective operation proportion in the inspiration period is calculated based on the effective operation number for the bronchus in the inspiration period in the set time period and the second total operation number, and the airway wall collision frequency in the expiration period is calculated based on the airway wall collision number for the bronchus in the expiration period in the set time period and the third total operation number. Then, the effective operation proportion and the airway wall collision frequency are weighted and summed according to a preset weighting ratio to obtain the respiratory adaptation degree. The set weighting ratio can be set according to the scene demand of the respiratory adaptation degree. For example, assuming that the effective operation proportion is 60%, the corresponding weight is 0.6, and the airway wall collision frequency is 30%, the corresponding weight is 0.4, then the respiratory adaptation degree can be 0.6 x 0.6-0.3 x 0.4=0.24.

[0059] Then, the fine branch lesion treatment precision is calculated based on the success rate of the fine bronchus for the bronchus in the set time period and the block-shaped sputum residue rate. In one example, the average value of the hemostasis success rate and the block-shaped sputum residue rate can be calculated as the fine branch lesion treatment precision. For example, assuming that the hemostasis success rate is 90% and the block-shaped sputum residue rate is 80%, the fine branch lesion treatment precision is 85%.

[0060] The score determination unit is configured to perform weighted fusion on the recognition accuracy, the respiratory adaptation degree, and the fine branch lesion treatment precision by a weighted fusion algorithm to obtain a comprehensive ability score. The comprehensive ability score is a quantitative result for reflecting the overall ability of the target operator, and the higher the value is, the stronger the operation ability is.

[0061] In one example, the score determination unit can perform the following steps.

[0062] First, the recognition accuracy, respiratory adaptation, and small branch lesion processing accuracy are normalized to obtain normalized data. Normalization refers to an operation that eliminates differences in the dimensions of different indicators for recognition accuracy, respiratory adaptation, and small branch lesion processing accuracy. For example, it is uniformly mapped to the interval [0, 1]. In one example, the normalized value = (original value - minimum value of the indicator) ÷ (maximum value of the indicator - minimum value of the indicator). The minimum value of the recognition accuracy and the small branch lesion processing accuracy can be 0, and the maximum value can be 100%. The minimum value of the respiratory adaptation can be 0, and the maximum value can be 1.

[0063] Then, based on the set clinical logic, the preset allocation weight of the normalized data is obtained. The preset weight is a weight parameter set based on the clinical operation logic. For example, navigation is the basis for bronchoscopy operation, respiratory coordination operation is the safety core, and treatment is the ultimate goal. Therefore, the operation priority included in the clinical logic can be navigation, operation, and treatment in turn, and the weight of navigation can be greater than that of operation and treatment.

[0064] Next, the normalized data is weighted and fused based on the preset allocation weight to obtain a comprehensive ability score. Comprehensive ability score = (recognition accuracy normalized value x weight 1) + (respiratory adaptation normalized value x weight 2) + (small bronchial processing accuracy normalized value x weight 3). Weight 1 corresponds to recognition accuracy, weight 2 corresponds to respiratory adaptation, and weight 3 corresponds to small bronchial processing accuracy.

[0065] The normalization process eliminates differences between indicators, ensuring the scientificity of the weighted fusion. The weight allocation matches the clinical operation logic, improving the rationality of the evaluation. In this way, both the overall operation ability level and the core ability can be highlighted, providing a data basis for hierarchical training.

[0066] The weak point determination unit is used to input the recognition accuracy, respiratory adaptation, and small branch lesion into the decision tree model to determine the weak point of ability.

[0067] The decision tree model is a hierarchical identification model designed for bronchoscopy. The nodes of the decision tree can be constructed based on anatomical features and physiological features. The weak point of ability is located by the decision tree and is a specific shortcoming that is strongly associated with clinical errors.

[0068] In one example, the weak point determination unit can perform the following steps.

[0069] Firstly, the total error number of core operation links of the bronchus in a set time period is taken as a root node, a plurality of target scenes of the virtual bronchoscope are taken as a plurality of first-level child nodes, and an anatomical scene of the virtual bronchoscope is taken as a plurality of second-level child nodes, to construct a decision tree model.

[0070] The core operation link is a key step prone to error in bronchoscope operation, which can include branch identification, respiratory coordination operation, and small bronchus lesion treatment. The set time period can also be determined according to a training period, and the total error number of the core operation link in the set time period is taken as the root node, and then the first-level child nodes and the second-level child nodes are further constructed. The target scene of the virtual bronchoscope refers to a scene corresponding to different operations, and each target scene can correspond to an operation type. For example, the error number of branch identification, the collision number in the respiratory period, and the error number of small bronchus lesion treatment. Further, the anatomical scene of the virtual bronchoscope is obtained according to anatomical characteristics and physiological characteristics. For example, the error number of branch identification can be subdivided into tracheal carina identification error, left lower lobe bifurcation identification error, and right middle lobe stenosis segment identification error. The leaf node can be a weak point of judgment.

[0071] After the decision tree model is constructed, the operation original data of the target operator can be input into the decision tree model to obtain the weak point of ability. The operation original data are all associated with bronchoscope anatomical or physiological characteristics, and are taken as input data of the decision tree model. Starting from the root node, the first-level child nodes are matched layer by layer, the second-level child nodes are synchronously analyzed, and then the judgment result of the leaf node is output, so that the weak point of ability of the target operator can be obtained.

[0072] The decision tree node of the embodiment of the application is designed based on the characteristics of the bronchoscope, which can accurately locate the specific branch and the refined short board of the respiratory phase, so as to accurately locate the weak point of ability of the target operator and improve the efficiency of subsequent targeted training.

[0073] In the embodiment of the application, the adjustment module 203 can include a first matching unit, a second matching unit, and a third matching unit.

[0074] The first matching unit is configured to match a first training scene of a novice level in a case where the comprehensive ability score is less than a first set score. The first set score is a threshold value for defining the ability of a novice. The first training scene is a basic operation scene designed for the novice level, which focuses on the entry-level core ability of the bronchoscope. Therefore, the first training scene can include a training scene for operating the first region between the main bronchus and the lung lobe bronchus, a training scene of lavage operation, and a training scene of virtual instrument operation based on a flexible scope, and the force feedback threshold value of the VR device 3 is a first feedback threshold value.

[0075] The first region is the region between the main bronchus and the lobar bronchus, with a diameter greater than 10 mm, few branches and large space, suitable for navigation entry. The basic treatment operation of flushing the airway through the virtual lavage tube during lavage operation is the most commonly used entry-level treatment means. The flexible scope is a flexible bronchoscope with a simulated diameter of 5 to 6 mm, which is relatively low in operation difficulty and suitable for beginners to familiarize with the mirror turning. The first feedback threshold is the resistance value fed back by the VR device 3 of the beginner level. For example, it can be set to 0.2 N.

[0076] Specifically, when the comprehensive ability score is less than a first set score, such as 60 points, the first matching unit is activated. The training scene only opens large-diameter branch regions such as bronchial segments, right upper lobe bronchus and left upper lobe bronchus, and shields complex bifurcations. Then set the operation tasks of tracheal carina recognition, lobar bronchus navigation and relationship operation, and limit to provide only flexible scope and basic lavage tube, simplify the difficulty of instrument selection. When the pushing force exceeds the first feedback threshold, a prompt is given, leaving room for trial and error. The above configuration is sent to the VR device 3 to generate the first training scene of the beginner level. See Figure 3 , Figure 3 A schematic diagram of a first training scene provided in an embodiment of the present application.

[0077] The second matching unit is used to match the second training scene of the skilled level when the comprehensive ability score is greater than or equal to the first set score and less than a second set score. The second set score is a threshold that defines the ability of the skilled level. The second training scene is an advanced scene designed for the skilled level to strengthen the ability to operate in complex regions and cooperate with multiple types of instruments. Therefore, the second training scene can include a training scene for operating in the second region between the pulmonary segments and the sub-segment bronchus, a training scene for forceps operation, and a training scene for virtual instrument operation of rigid scope based on flexible scope and target range, and the force feedback threshold of the VR device 3 is a second feedback threshold.

[0078] The second region is the region between the pulmonary segments and the sub-segment bronchus, with a diameter of 5-10 mm, more branches and complex angles, suitable for advanced training. The forceps operation is a middle-level treatment operation of grabbing airway foreign matter or tissue by virtual biopsy forceps. The target range rigid scope is a rigid scope suitable for the advanced stage, for example, a rigid scope with a diameter of 4-5 mm, which is only adapted to specific branches. The second feedback threshold is the resistance value fed back by the VR device 3 of the advanced level. The skilled stage has more stringent requirements for operation accuracy than the beginner level, so the second feedback threshold is less than the first feedback threshold. For example, it can be set to 0.15.

[0079] Specifically, when the comprehensive ability score is greater than or equal to the first set score but less than the second set score, such as greater than or equal to 60 points but less than 80 points, the second matching unit is activated. The training scene opens the area of the lung segment to the sub-segment bronchus, such as the complex anatomic structure that can include the bifurcation of the left lower lobe basal segment, the narrow segment of the right middle lobe, etc. Then set the complex branch navigation, foreign body forceps taking and narrow segment passing and other composite tasks, and provide flexible mirror plus a certain amount of rigidity. The target operator needs to select the appropriate instrument according to the branch diameter for operation. When the pushing force exceeds the set second feedback threshold, vibration prompt can be performed. The above configuration is sent to the VR device 3 to generate the second training scene of the skilled level. See Figure 4 , Figure 4 FIG. 3 is a schematic diagram of a second training scene provided in an embodiment of the present application.

[0080] The third matching unit is used to match the third training scene of the expert level when the comprehensive ability score is greater than or equal to the second set score. The third training scene is a high-difficulty scene designed for the expert level, focusing on the ability of peripheral small bronchus treatment and emergency handling. Therefore, the third training scene can include training scenes for operating in the third area of small bronchus and mixed lesions, biopsy operation, and virtual instrument operation based on flexible mirror and full-size rigid mirror. The force feedback threshold of the VR device 3 is the third feedback threshold.

[0081] The third area is an area composed of small bronchus and mixed lesions. Biopsy operation is a high-level treatment operation for obtaining peripheral lesion tissue through virtual biopsy forceps, which needs to avoid blood vessels and has higher precision requirements. In the third training scene, full-size rigid mirror can be covered. And the third feedback threshold is the resistance value fed back by the expert-level VR device 3. For example, it can be set to 0.1 N.

[0082] Specifically, when the comprehensive ability score is greater than or equal to the second set score, such as 80 points, the third matching unit is activated. The training scene opens the small bronchus segment and superimposes the mixed lesions. Peripheral branch navigation, biopsy, hemostasis, sputum suction, and narrow segment expansion emergency tasks are set as operation tasks. At the same time, all instrument selections are opened. When the pushing force exceeds the set third feedback threshold, prompt is performed. Since the training precision requirement of the expert level is higher, the third feedback threshold is less than the second feedback threshold. See Figure 5 , Figure 5 FIG. 4 is a schematic diagram of a third training scene provided in an embodiment of the present application.

[0083] In the embodiments of the present application, the adjustment module 203 can also include a fourth matching unit, a fifth matching unit, a sixth matching unit, and a reminding unit.

[0084] The fourth matching unit is configured to assign a leading role to the target operator. The leading role is the core role of the multi-role collaborative training, which can replicate the responsibilities of the leading clinician in a bronchoscopy operation. The training task of the leading role can include controlling a virtual bronchoscope through the VR device 3, being responsible for the advancement of the scope and navigation of multiple branches, and the display interface of the VR device 3 marking the names of the bronchial branches in real time. In one example, the target operator with the highest comprehensive ability score can be assigned the leading role, and a simplified bronchial tree map can be provided on the VR display interface of the leading role, with the path highlighted and without specific operation step prompts, so that the leading role makes autonomous decisions.

[0085] The fifth matching unit is configured to assign an assistant role to the target operator. The assistant role is responsible for the responsibilities of a clinical assistant, focusing on lesion treatment and operation execution, and assisting the leading role. The training task of the assistant role can include responding to the virtual bronchoscope controlled by the leading role to reach a preset lesion point, and calling a virtual instrument corresponding to the lesion point through the VR device within a set time for cooperative operation, and the operation result will be synchronized to the leading role or the instrument nurse role. If the operation is not completed within the timeout or the instrument is selected incorrectly, the VR interface will prompt a cooperation failure, and the leading role needs to re-navigate and make decisions. Through the simulation of the cooperation scenario of the leading role instruction and the assistant execution, the reaction ability and execution precision of the target operator can be improved. The fault tolerance mechanism can cultivate the clinical thinking of active cooperation and non-passive execution.

[0086] The sixth matching unit is configured to assign an instrument nurse role to the target operator. The instrument nurse role is to replicate the responsibilities of a clinical nurse, focusing on instrument preparation and adaptation judgment. The training task of the instrument nurse role can include selecting an instrument based on a virtual bronchoscope instrument library according to the operation of the leading role. The VR display interface of the instrument nurse role can display a classified instrument library, real-time mark the adaptation parameters of each instrument, and when the leading role advances to the corresponding node, select and virtually deliver the instrument. Moreover, the execution time of the selected instrument can be timed to determine the operation efficiency of the role.

[0087] The reminding unit is configured to generate a breathing waveform in real time through the VR device 3, which is a dynamic waveform graph of the display interface of the VR device 3, synchronized with the real breathing frequency. The leading role is prompted to perform branch switching during the inspiration period of the breathing waveform, and the VR device is triggered to vibrate as a warning and pause the operation during the expiration period of the breathing waveform, and the prompt information is synchronized to the display interfaces of the assistant role and the instrument nurse role. Through waveform visualization and multi-modal warning, all roles can follow the clinical specifications of inspiration period operation and expiration period pause, solve the problem of airway injury caused by disconnection between breathing and operation, improve the fluency of multi-role cooperation, and improve the team operation efficiency.

[0088] In the embodiment of the present application, the interaction module 204 can include a labeling unit, an amplification unit and a simulation unit.

[0089] The labeling unit is used to receive the operation suggestions marked by the teacher through the AR device 4, and superimpose the AR marking line in the operation suggestion to the display interface of the VR device 3. The operation suggestion is the guidance information related to the virtual bronchial operation input by the teacher through the AR device 4. The AR marking line is a virtual line drawn through the AR device 4, which is used to intuitively mark the operation path, key area or error position, etc. The original coordinates of the AR marking line are converted into the scene coordinate system of the VR device 3 through the multi-modal input of the teacher such as gesture marking or voice instruction through the AR device 4, so as to send the processed AR marking line data to the VR device 3, and superimpose in real time in the use scene of the virtual bronchoscope 2.

[0090] The amplification unit is used to amplify the display area of the target object in the display interface of the VR device 3 based on the target object of the anatomical task when the operation task exists. The anatomical task is related to the anatomical recognition or operation task in the virtual training. The target object is a specific bronchial structure focused on the anatomical task, which is a high-frequency difficulty point for recognition. Therefore, the display size of the target object can be adjusted by adjusting the display parameters of the VR device 3, while the surrounding structure is displayed in a reduced size to reduce the occlusion. In this way, the complex anatomical target can be dynamically amplified, so that the operator can clearly observe the key features such as branch angle and pipe diameter difference, and the recognition error rate is reduced.

[0091] The simulation unit is used to simulate the operation path based on the anatomical suggestion when the operation suggestion exists. The anatomical suggestion is a guidance for the operation steps for a specific anatomical scene. The simulation operation path is a virtual operation demonstration animation generated based on the anatomical suggestion, which can intuitively show the correct steps through the motion trajectory of the virtual bronchoscope 2, and can have a time sequence mark. For example, color marks, text prompts, etc. can be added to the simulation path. In this way, the abstract steps can be converted into observable and imitable dynamic processes through visual animation.

[0092] In addition, in the embodiment of the present application, the AR device 4 configured for teaching can not be limited by the region, and can be remotely connected with the controller 1, and then the operation suggestion is synchronized to the display interface of the VR device 3 through the controller 1, so as to remotely guide the target operator in the training, thereby solving the problem that the teaching resources are limited by the region.

[0093] Figure 6 A structure schematic diagram of a virtual bronchoscope teaching system 200 provided in another embodiment of the present application is shown in FIG. 2. Figure 6 As shown in the embodiment of the present application, the virtual bronchoscope teaching system 200 can further include an alarm module 205 and a traceability module 206.

[0094] The alarm module 205 is configured to trigger an alarm prompt of the VR device 3 and display an emergency operation suggestion on the display interface of the VR device when it is detected that there is an operation risk.

[0095] The operation risk refers to a high-risk behavior that the target operator may cause in the virtual bronchoscope operation, such as virtual airway injury, operation failure or simulated complications. When it is detected that there is an operation risk, the VR device 3 can perform alarm through multi-modal output of risk prompts, such as sound, touch and vision. For example, through strong vibration of the handle, flickering of the interface frame and warning sound, etc. And it can also make emergency operation suggestions for specific risks, and replicate the clinical first aid process. For example, through text, dynamic arrow, etc. superimposed on the display interface of the VR device 3. In this way, the emergency response of the clinical high-risk scene can be simulated, the target operator can be helped to quickly make risk prediction and first aid measures, and the clinical emergency handling ability can be improved.

[0096] The traceability module 206 is configured to generate an operation report based on the historical operation data of the target operator, and the operation report is associated with the risk operation of the target operator and the lung function index of the operation object.

[0097] The historical operation data is the full amount of data generated by the target operator in the virtual training, which can include multi-modal operation data, risk operation record and training scene parameter, etc. The operation report is a structured document generated based on the historical operation data, which can include a capability trend chart, a risk operation heat map and a lung function correlation analysis. In the virtual scene, the influence of the risk operation on the simulated lung function can be mapped to associate the risk operation with the lung function index, which can include virtual ventilation volume, airway resistance, blood oxygen saturation, etc. In this way, the training process of each target operator can be traced, and the training effect can be quantified. And the target operator can clearly understand the specific consequences of the risk operation, shorten the teaching cycle and improve the teaching efficiency.

[0098] The above application uses specific examples to illustrate the present application, which is only used to help understand the present application and does not limit the present application. For those skilled in the art to which the present application belongs, according to the idea of the present application, a number of simple deductions, deformations or substitutions can be made.

Claims

1. A virtual teaching system for bronchoscopy, characterized in that, The virtual teaching system is in communication with a VR device and an AR device, and comprises: an acquisition module configured to acquire, by the VR device, multi-modal operation data of a target operator on a virtual bronchoscope collected in real time; an evaluation module configured to generate an ability profile of the target operator and determine weak points in the ability of the target operator based on the multi-modal operation data, in combination with multi-level branch anatomical features and respiratory physiological features of a bronchus; an adjustment module configured to select training content matching the ability profile and the weak points in the ability from a training library according to the ability profile and the weak points in the ability, the training content including single-role hierarchical training, multi-role collaborative training, and weak point reinforcement training; an interaction module configured to acquire, in a training process, operation suggestions marked by a teacher through the AR device, adjust display parameters of the VR device based on the operation suggestions, and update a display interface of the VR device based on the display parameters; wherein the acquisition module comprises: a first acquisition unit configured to acquire identification parameters of the target operator on multi-level branches of a bronchus, the identification parameters including correct times and first total operation times of virtual bronchoscope operation on carina of the bronchus, bifurcation of the lower basal segment of the left lung, and the narrow segment of the middle lobe of the right lung; a second acquisition unit configured to acquire respiratory operation parameters of the target operator at multiple respiratory phases, the respiratory operation parameters including effective operation times and second total operation times of the bronchus in the inspiration phase, and airway wall collision times and third total operation times of the bronchus in the expiration phase; a third acquisition unit configured to acquire lesion processing parameters of the target operator on lesions, the lesion processing parameters including a success rate of hemostasis and a residual rate of massive sputum on small bronchial lesions.

2. The virtual teaching system of claim 1, wherein, The evaluation module comprises: an index calculation unit configured to calculate an identification accuracy, a respiratory adaptation degree, and a small branch lesion processing precision based on the multi-modal operation data; a score determination unit configured to obtain a comprehensive ability score by weighted fusion of the identification accuracy, the respiratory adaptation degree, and the small branch lesion processing precision through a weighted fusion algorithm; a weak point determination unit configured to input the identification accuracy, the respiratory adaptation degree, and the small branch lesion processing precision into a decision tree model to determine the weak points in the ability.

3. The virtual teaching system of claim 2, wherein, The index calculation unit is configured to: calculate a proportion of correct times of virtual bronchoscope operation on carina of the bronchus, bifurcation of the lower basal segment of the left lung, and the narrow segment of the middle lobe of the right lung in the first total operation times in a set time period to obtain the identification accuracy; calculate an effective operation proportion in the inspiration phase based on effective operation times and second total operation times of the bronchus in the inspiration phase in the set time period, and calculate an airway wall collision frequency in the expiration phase based on airway wall collision times and third total operation times of the bronchus in the expiration phase in the set time period; weight and sum the effective operation proportion and the airway wall collision frequency according to a preset weighting ratio to obtain the respiratory adaptation degree. The success rate and the blocky sputum residual rate of the small bronchus of the bronchus in the set time period are used to calculate a small branch lesion treatment accuracy.

4. The virtual teaching system of claim 3, wherein, The score determination unit is configured to: normalize the recognition accuracy, the breathing adaptation degree, and the small branch lesion treatment accuracy to obtain normalized data; based on a set clinical logic, obtain a preset allocation weight of the normalized data, the operation priority of the clinical logic including navigation, operation, and treatment in sequence; based on the preset allocation weight, perform weighted fusion calculation on the normalized data to obtain the comprehensive ability score.

5. The virtual teaching system of claim 3, wherein, The weak point determination unit is configured to: construct a decision tree model with the total number of errors of the core operation link of the bronchus in the set time period as a root node, a plurality of target scenes of the virtual bronchoscope as a plurality of first-level child nodes, and an anatomical scene of the virtual bronchoscope as a plurality of second-level child nodes; input operation raw data of the target operator into the decision tree model to obtain the weak point, the operation raw data being associated with bronchoscope anatomy or physiological characteristics.

6. The virtual teaching system of claim 1, wherein, The adjustment module includes: a first matching unit configured to, in a case where the comprehensive ability score is less than a first set score, match a first training scene of a novice level, the first training scene including a training scene for operation on a first region from a main bronchus to a lobar bronchus, a training scene of a lavage operation, and a training scene of a virtual instrument operation based on a flexible scope, a force feedback threshold of the VR device being a first feedback threshold; a second matching unit configured to, in a case where the comprehensive ability score is greater than or equal to the first set score and less than a second set score, match a second training scene of a skilled level, the second training scene including a training scene for operation on a second region from a pulmonary segment to a sub-segment bronchus, a training scene of a forceps operation, and a training scene of a virtual instrument operation based on the flexible scope and a rigid scope of a target range, a force feedback threshold of the VR device being a second feedback threshold, the second feedback threshold being less than the first feedback threshold; a third matching unit configured to, in a case where the comprehensive ability score is greater than or equal to the second set score, match a third training scene of an expert level, the third training scene including a training scene for operation on a third region of a small bronchus and a mixed lesion, a training scene of a biopsy operation, and a training scene of a virtual instrument operation based on the flexible scope and a full-size rigid scope, a force feedback threshold of the VR device being a third feedback threshold, the third feedback threshold being less than the second feedback threshold.

7. The virtual teaching system of claim 1, wherein, The adjustment module includes: a fourth matching unit configured to assign a main operator role to the target operator, the training task of the main operator role including controlling a virtual bronchoscope through the VR device, being responsible for mirror body advancement, multi-level branch navigation, and real-time labeling of bronchial branch names on a VR device display interface. The fifth matching unit is configured to assign an assistant role to the target operator, and the training task of the assistant role includes sending a virtual bronchoscope controlled by the main operator to a preset lesion point and calling a virtual instrument corresponding to the lesion point through the VR device for cooperation operation within a set time. The sixth matching unit is configured to assign an instrument nurse role to the target operator, and the training task of the instrument nurse role includes selecting an appropriate instrument based on a virtual bronchoscope instrument library according to the operation of the main operator. The reminding unit is configured to generate a breathing waveform in real time through the VR device, prompt the main operator to perform branch switching during the inspiration period of the breathing waveform through different prompt information, trigger vibration warning and operation pause prompt of the VR device during the expiration period of the breathing waveform, and synchronize the prompt information to the display interfaces of the assistant role and the instrument nurse role.

8. The virtual teaching system of claim 1, wherein, The interaction module comprises: The labeling unit is configured to receive operation suggestions marked by a teacher through the AR device, and superimpose AR marking lines in the operation suggestions to the display interface of the VR device; The magnification unit is configured to magnify a target object in the display area of the display interface of the VR device based on the target object of the dissection task when the operation task includes the dissection task; The simulation unit is configured to perform simulation operation on the operation path based on the dissection suggestion when the operation suggestion includes the dissection suggestion.

9. The virtual teaching system of claim 1, wherein, Further comprising: The alarm module is configured to trigger an alarm prompt of the VR device when it is detected that there is an operation risk, and display an emergency operation suggestion on the display interface of the VR device; The traceability module is configured to generate an operation report based on the historical operation data of the target operator, and the operation report is associated with the risk operation of the target operator and the lung function index of the operation object.

Images