Ear-nose-throat operation teaching method and system based on virtual reality

Abstract

The invention provides an ear-nose-throat operation teaching method and system based on virtual reality, and the method comprises the steps: carrying out the matching of an optimal teaching material based on the historical record of ear-nose-throat operation learning of a user; generating a virtual reality model based on the optimal teaching material; guiding the user to enter the virtual reality model; and after the user enters the virtual reality model, controlling continuous teaching interaction between the virtual reality model and the user. According to the invention, students can enter virtual reality simulation at any time to carry out ear-nose-throat surgery operation simulation in the process of waiting for a real surgery practical operation opportunity, so that the requirements of the students are met; on the basis of historical records of ear-nose-throat operation learning of the user, optimal teaching materials are matched, personalized design of teaching content is achieved, and the teaching effect on students is improved; students enter virtual reality simulation to carry out ear-nose-throat operation simulation without being limited by sites and the like, so that the skill improvement convenience of the students is improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to an ear, nose and throat surgery teaching method and system based on virtual reality. Background Art

[0002] Currently, in the field of otolaryngology, traditional surgical teaching usually relies on face-to-face master-apprentice teaching, surgical demonstrations and simulated operations.

[0003] However, this traditional teaching model has certain limitations: due to the high risk and complexity of surgical practice, the actual surgical opportunities that students can participate in are usually limited and often concentrated on a few advanced students, making it difficult to meet the needs of most students; in addition, traditional teaching content is often fixed and lacks personalized design, and cannot be flexibly adjusted according to the different foundations and learning progress of students, thus affecting the teaching effect.

[0004] On the other hand, traditional surgical teaching emphasizes repeated practice of surgical skills. However, due to factors such as venue and time constraints, trainees can often only conduct practical training under limited conditions. This type of practical training method not only fails to provide sufficient practice opportunities, but also leads to greater constraints on trainees in the process of improving their skills.

[0005] Therefore, a solution is urgently needed. Summary of the invention

[0006] One of the purposes of the present invention is to provide an ENT surgery teaching method based on virtual reality. Trainees can enter the virtual reality simulation at any time to perform ENT surgery operation simulation while waiting for the opportunity to practice real surgery, so as to meet the needs of trainees; based on the user's historical records of ENT surgery learning, the best teaching materials are matched to realize personalized design of teaching content and improve the teaching effect for trainees; trainees enter the virtual reality simulation to perform ENT surgery operation simulation without being restricted by the venue, etc., which improves the convenience of trainees in improving their skills.

[0007] An embodiment of the present invention provides a virtual reality-based ENT surgery teaching method, comprising: Match the best teaching materials based on the user's history of learning ENT surgery; Generate virtual reality models based on the best teaching materials; Guide users into the virtual reality model; After the user enters the virtual reality model, the virtual reality model is controlled to continuously perform teaching interaction with the user.

[0008] Optionally, controlling the virtual reality model to continuously perform teaching interaction with the user includes: Control the virtual reality model to start teaching the user; Determine multiple target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the multiple target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress; When the virtual reality model teaches the user and enters any target teaching progress, it continuously identifies whether the user has negative emotions; If so, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user are determined to determine the time period and strategy for improving the emotions; When the virtual reality model teaches the user to enter a period of mood improvement, the user's negative mood is improved based on the mood improvement strategy.

[0009] Optionally, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user, determining the time period for improving emotions and the emotion improvement strategy, includes: If the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user learning the remaining teaching content; Otherwise, the pre-teaching gap of the first remaining teaching content, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content in the remaining teaching content sequence are taken as the opportunity periods for mood improvement; Determine target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; Generate a target timeline of times when mood improves; Content output control mechanism to plan the output content sequence of target improvement knowledge on the target timeline; Determine the mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

[0010] Optionally, the content output control mechanism for planning the output content sequence of the target improvement knowledge on the target timeline includes: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies a local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

[0011] Optionally, the local sequence constraint includes: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

[0012] An embodiment of the present invention provides an ENT surgery teaching system based on virtual reality, comprising: The material matching module is used to match the best teaching material based on the user's history of learning ENT surgery; Model generation module, used to generate virtual reality models based on the best teaching materials; An entry guidance module is used to guide the user to enter the virtual reality model; The interactive control module is used to control the virtual reality model to continuously perform teaching interaction with the user after the user enters the virtual reality model.

[0013] Optionally, the interaction control module controls the virtual reality model to continuously perform teaching interaction with the user, including: Control the virtual reality model to start teaching the user; Determine multiple target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the multiple target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress; When the virtual reality model teaches the user and enters any target teaching progress, it continuously identifies whether the user has negative emotions; If so, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user are determined to determine the time period and strategy for improving the emotions; When the virtual reality model teaches the user to enter a period of mood improvement, the user's negative mood is improved based on the mood improvement strategy.

[0014] Optionally, the interactive control module determines the time period and strategy for improving emotions based on the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the user's bad emotions, including: If the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user learning the remaining teaching content; Otherwise, the pre-teaching gap of the first remaining teaching content, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content in the remaining teaching content sequence are taken as the opportunity periods for mood improvement; Determine target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; Generate a target timeline of times when mood improves; Content output control mechanism to plan the output content sequence of target improvement knowledge on the target timeline; Determine the mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

[0015] Optionally, the interactive control module plans a content output control mechanism for the output content sequence of the target improvement knowledge on the target timeline, including: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies a local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

[0016] Optionally, the local sequence constraint includes: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0018] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 Schematic diagram of a virtual reality-based ENT surgery teaching method according to an embodiment of the present invention; Figure 2 Schematic diagram of an ENT surgery teaching system based on virtual reality in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0021] The embodiment of the present invention provides an ENT surgery teaching method based on virtual reality, such as Figure 1 As shown, including: S1. Match the best teaching materials based on the user's history of learning ENT surgery; In S1, the system will collect the user's historical records during the ENT surgery learning process, including: the surgical skills that the user has mastered, learning progress, difficulties encountered, preferences, and past learning effects; the historical records can come from traditional teaching platforms, surgical simulators, and the user's historical learning data in a virtual reality environment; by analyzing the user's historical records, the system can intelligently recommend the teaching materials that best match the user's current learning needs, that is, the best teaching materials. For example, if a user showed difficulty in ear surgery in the last study, the system can use ear surgery operation models and learning materials as the best teaching materials to help the user improve in a targeted manner; S2, generate virtual reality models based on the best teaching materials; In S2, the virtual reality model can include 3D anatomical structures of the ear, nose and throat, surgical instruments, surgical environment and other elements, which are determined by the best teaching materials; when generating the virtual reality model, it can be realized based on three-dimensional modeling technology; in the virtual reality model, the user can simulate the surgical operation; S3, guiding the user to enter the virtual reality model; In S3, users can wear VR devices (VR headsets, etc.) to enter the virtual reality model and perform surgical operation simulations; S4. After the user enters the virtual reality model, the virtual reality model is controlled to continuously perform teaching interaction with the user.

[0022] In S4, when controlling the continuous interaction between the virtual reality model and the user, for example, if the user encounters problems during the surgical operation, the system can provide immediate guidance and remind the user to pay attention to certain operation details; for example, through the sensor data of the VR device (such as the user's hand movements, eye gaze points, hand posture changes, etc.), the system can monitor in real time whether the user's operations comply with medical standards and give feedback.

[0023] While waiting for the opportunity to perform real surgery, students of this application can enter the virtual reality simulation at any time to perform ENT surgery simulation, which meets the needs of students; based on the user's historical records of ENT surgery learning, the best teaching materials are matched to achieve personalized design of teaching content and improve the teaching effect for students; students entering the virtual reality simulation to perform ENT surgery simulation are not restricted by venues, which improves the convenience of students improving their skills.

[0024] In one embodiment, in S4, controlling the virtual reality model to continuously perform teaching interaction with the user includes: S41, controlling the virtual reality model to start teaching the user; In S41, when the virtual reality model is controlled to start teaching the user, the system will prompt the user in the virtual reality model how to start the surgical operation simulation, etc.; S42, determining a plurality of target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the plurality of target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress of the teaching; In S42, when the virtual reality model teaches the user, it will formulate a teaching progress sequence, which is determined by the best teaching material. There are multiple teaching progress sequences for teaching the user in sequence, and the remaining teaching progress sequence contains the remaining teaching progress that needs to be taught to the user. When students are learning otolaryngology surgery, even in a virtual environment, they may experience negative emotions such as tension, anxiety, loss of control, and self-doubt. The possibility threshold can be 8. When the possibility is greater than or equal to the possibility threshold, it means that the user may have a greater degree of negative emotions when the virtual reality model teaches the user to enter the corresponding teaching progress. However, the virtual reality model teaches the user to enter the previous teaching progress of the remaining teaching progress with a possibility greater than or equal to the possibility threshold. When there are a number of remaining teaching progress, the user may feel that he or she will have bad emotions in the next teaching progress and may also have bad emotions in the current teaching progress. Therefore, the remaining teaching progress with a probability greater than or equal to the probability threshold in the remaining teaching progress sequence and the previous remaining teaching progress with a probability greater than or equal to the probability threshold are taken as the target teaching progress; only within the target teaching progress is the identification of whether the user has bad emotions, which reduces the identification resources of the system and improves the identification efficiency of the system; when determining the probability of the remaining teaching progress, the technician can make a comprehensive setting in advance based on the difficulty of learning the remaining teaching progress, the user's operating experience, and the frequency of bad emotions in the user's historical surgical simulation operations; S43, when the virtual reality model is teaching the user and entering any target teaching progress, continuously identifying whether the user has negative emotions; In S43, when identifying whether the user has a bad emotion, the sensor data of the VR device worn by the user can be analyzed to determine, for example: the user's hand shaking causes continuous instability, indicating that the user is nervous; for another example: the user's facial expression is solemn, indicating that the user is nervous; for another example: the user is hesitant when performing a certain surgical simulation operation, indicating that the user has self-doubt, etc. S44, if yes, determine the time period and strategy for improving emotions based on the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the user's bad emotions; In S44, the knowledge base for improving bad emotions contains a large amount of improvement knowledge, which indicates how to improve bad emotions and can be the experience of other students in overcoming bad emotions in history, etc.; the target teaching progress has a remaining teaching content sequence, and the remaining teaching content sequence contains multiple teaching contents that need to be taught to users in sequence; the remaining teaching content sequence reflects the remaining content of the virtual reality model for teaching users, and the improvement knowledge base provides improvement knowledge. Therefore, the emotion improvement opportunity period and emotion improvement strategy can be determined based on the two; the emotion improvement opportunity is the opportunity to improve the bad emotions of users in the future, and the emotion improvement strategy is the strategy to improve the bad emotions of users in the future; S45. When the virtual reality model teaches the user to enter a mood improvement opportunity period, based on the mood improvement strategy, improve the user's negative mood.

[0025] In S45, finally, when entering the mood improvement opportunity period, based on the mood improvement strategy, the user's negative mood is improved to ensure that the user's mood is improved with appropriate means at an appropriate time.

[0026] When the embodiment of the present invention controls the virtual reality model to continuously interact with the user for teaching, it specifically identifies whether the user has bad emotions within the target teaching progress. If so, it determines the mood improvement opportunity period and mood improvement strategy in a timely, accurate and comprehensive manner, and improves the user's mood based on the mood improvement strategy when entering the mood improvement opportunity period. The user does not need to stop and continue the ENT surgery simulation, but can take off the VR device and then ask other students or instructors how to overcome bad emotions to avoid delaying the learning progress, which greatly improves the convenience and humanization. When the user has bad emotions, the degree of bad emotions will become more and more serious, and the user can be helped to improve in time. The method of improving bad emotions can be most deeply remembered, which is convenient for the user to use in the subsequent same ENT surgery simulation or actual surgery, thereby improving the ENT surgery skill level.

[0027] In one embodiment, the step S44, determining the time period for improving emotions and the emotion improvement strategy based on the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user, comprises: S441, if the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user to learn the remaining teaching content; The importance threshold can be 5; when the importance is greater than or equal to the importance threshold, it means that the importance of the corresponding remaining teaching content is relatively large; in the remaining teaching content sequence, there is a teaching gap before the first remaining teaching content and between two remaining teaching contents, and the teaching gap is the time left for the user to review, rest, and freely operate after the remaining teaching content is taught; when the importance of the first remaining teaching content is greater than or equal to the importance threshold, it means that the importance of the remaining teaching content to be taught to the user is relatively large, and if the user's bad emotions are brought into the teaching process of the remaining teaching content, it will affect the teaching effect. Therefore, before the teaching of the remaining teaching content, it is necessary to try to overcome the bad emotions. Therefore, the teaching gap before the first remaining teaching content in the remaining teaching content sequence is used as an opportunity period for emotion improvement; the importance of different remaining teaching contents can be comprehensively set by technical personnel in advance according to the learning difficulty and learning time of the teaching content; S442, otherwise, the pre-teaching gap of the first remaining teaching content in the remaining teaching content sequence, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content are used as the mood improvement opportunity period; In S442, the teaching period is the time period during which the virtual reality model teaches the remaining teaching content; otherwise, that is, when the importance of the first remaining teaching content is less than the importance threshold, the teaching period of the first remaining teaching content can be used to improve the user's negative emotions. Secondly, in order to improve the user's negative emotions as soon as possible and avoid occupying the teaching of subsequent remaining teaching contents as much as possible, the post-teaching gap of the first remaining teaching content is also used to improve the user's negative emotions. Therefore, the pre-teaching gap of the first remaining teaching content in the remaining teaching content sequence, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content are used as the emotion improvement opportunity periods; S443, determining target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base having the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; In S443, in the improvement knowledge base, the improvement knowledge has a knowledge output requirement time, which is the time required for the improvement knowledge output to improve the user's negative emotions, and can be set by the technical staff in advance according to the degree of its improvement effect and the size of its content. The improvement knowledge with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity period in the improvement knowledge base is selected as the target improvement knowledge to improve the utilization rate of the emotion improvement opportunity period; the longer the output time of the improvement knowledge is, the better the effect of improving the user's negative emotions is. Therefore, selecting the target improvement knowledge in this way can also maximize the effect of improving the user's negative emotions. S444, generating a target timeline for the mood improvement opportunity period; In S444, a target time axis is generated, wherein the start time node of the target time axis is the start time of the mood improvement opportunity period, and the end time node of the target time axis is the end time of the mood improvement opportunity period; S445, a content output control mechanism for planning an output content sequence of target improvement knowledge on a target time axis; In S445, the target improvement knowledge has an output content sequence, and the output content sequence includes content that is output in sequence when the target improvement knowledge is output; the planned content output control mechanism instructs the system how to control the output content sequence for output; S446. Determine a mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

[0028] In S446 , after the output control mechanism completes the planning on the target timeline, it determines the emotion improvement strategy based thereon.

[0029] When determining the mood improvement opportunity period, the embodiment of the present invention accurately determines the method for determining the mood improvement opportunity period according to the importance of the first remaining teaching content in the remaining teaching content sequence, which greatly improves the suitability of determining the mood improvement opportunity period and further improves the applicability of the system; when using the improvement knowledge base, the most suitable target improvement knowledge is selected therefrom, and when using the target improvement knowledge, a content output control mechanism of the output content sequence of the target improvement knowledge is accurately planned on the target timeline, and the mood improvement strategy is determined based on the target timeline after the output control mechanism planning is completed, which greatly improves the adaptability of the emotion improvement strategy determination and ensures the effect of improving the user's negative emotions.

[0030] In one embodiment, the content output control mechanism of S445, planning the output content sequence of target improvement knowledge on the target time axis, includes: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies the local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

[0031] In the above technical scheme, under the constraint of local sequence constraints, multiple target local sequences are divided and determined in the output content sequence, and the output contents in the same target local sequence can be output simultaneously; the output content has a content output requirement time, and the content output requirement time is the time required for the output of the content to improve the user's negative emotions, which can be set in advance by technical personnel based on the degree of improvement effect and the size of the content. The sum of the output requirement times of different output contents in the output content requirement is the knowledge output requirement time of the corresponding improvement knowledge; in the output content sequence, the arrangement order of the output contents is the order in which they were originally intended to be output in sequence, therefore, each target local sequence is traversed in sequence according to the sequence order of each target local sequence in the output content sequence.

[0032] Each time when traversing, if the output content in the traversed target local sequence is unique, then based on the required time length of its content output, a first target time interval is set at the front of the front idle area on the target time axis, and the time length of the first target time interval is equal to the required time length of its content output. At the same time, a first control strategy is assigned to the first target time interval; in the first control strategy, the first target area is a content display area in the virtual reality model, and the first target area continuously displays the output content in the traversed target local sequence. The real-time gaze area of ​​the user in the virtual reality model refers to the area that the user gazes at in real time when performing simulated surgical operations in the virtual reality model, which can be obtained by wearing a VR device. The first standard position relationship represents a position relationship that allows the user to conveniently view the first target area containing a single output content while viewing the real-time gaze area, such as: the first target area is above or below the real-time gaze area (in a virtual environment, the up and down movement of the eyes can usually keep the user focused on a specific content, so it is more suitable for viewing a single output content), and the distance between the first target area and the real-time gaze area is 10 cm.

[0033] Otherwise, that is, if the output content in the traversed target local sequence is not unique, and the output content in the traversed target local sequence can be output simultaneously, then based on the maximum output demand duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the front idle area on the target time axis, and the time length of the second target time interval is equal to the maximum content output demand duration. At the same time, a second control strategy is assigned to the second target time interval; in the second control strategy, the second target area is also a content display area in the virtual reality model. The difference is that it can display multiple output contents at the same time, and it has a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model. The second standard position relationship represents the user in When checking the real-time gaze area, it is convenient to check the positional relationship of the second target area containing multiple output contents, such as: the second target area is on the left or right side of the real-time gaze area (when the user is in a virtual environment, the left and right movement of the eyes can make full use of the lateral range of vision, help the user to browse multiple information items more conveniently, and minimize visual fatigue and cognitive burden), and the distance between the second target area and the real-time gaze area is 15 cm; when the second target area outputs multiple output contents in the traversed target local sequence, it can be displayed in sequence according to the sequence order of the multiple output contents, ensuring that each output content has overlapping display time periods, and then ensuring that all output contents are displayed before the end of the second target time interval, thereby further improving the user experience.

[0034] After traversing each target local sequence, multiple first target time intervals and second target time intervals will be arranged adjacent to each other on the target time axis. When using the current target time axis, as the virtual reality model continues to teach the user, time continues to pass. If the teaching time enters the first target time interval, the first control strategy assigned to it will be executed. Similarly, if the teaching time enters the second target time interval, the second control strategy assigned to it will be executed to improve the user's negative emotions.

[0035] When planning the content output control mechanism on the target timeline, the embodiment of the present invention optimizes the output content sequence, divides it into multiple target local sequences, and uses the first target area to control the output and uses the second target area to control the output simultaneously according to whether the output content in the target local sequence is unique, so as to reasonably and accurately shorten the time for the system to intervene in improving the user's negative emotions, further avoid delaying the user's learning process, greatly improve the applicability of the system, and at the same time, it is more humane; secondly, the first target area continues to have a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model, and the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model, so that when the user views the real-time gaze area, it is very convenient to view the output content that needs to be output from the most suitable angle, thereby improving their learning experience in the virtual reality model and at the same time, it is more intelligent.

[0036] In one embodiment, the local sequence constraints include: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

[0037] In the above technical solution, in constraint one, there are two situations of output contents in the same target local sequence. The first situation is that it contains only one output content, and the second situation is that it contains at least two consecutive output contents. Continuous means that the output contents are consecutive in the output content sequence.

[0038] In constraint two, the standard content association relationship can be a logical association relationship of emotion regulation, operation guidance or operation steps. For example, in a surgical simulation, assuming that the user is uneasy when performing tumor resection, two output contents with a standard content association relationship can be: "Please stay calm, you are doing a good job. (Emotion regulation)" and "Next, you need to carefully cut the tissue around the tumor. (Operation guidance)". These two contents are continuous, and there is a relationship between emotional comfort and operation guidance between them, that is, the former helps relieve anxiety, and the latter provides specific operation suggestions; based on this, multiple output contents in the same target local sequence can be output simultaneously.

[0039] In constraint three, it is ensured that all output contents in the output content sequence are included in different target local sequences, so that the output contents are divided into different target local sequences.

[0040] The embodiment of the present invention sets three local sequence constraints, which improves the accuracy and suitability of dividing the output content sequence into multiple target local sequences, and greatly improves the applicability of the system.

[0041] The embodiment of the present invention provides an ENT surgery teaching system based on virtual reality, such as Figure 2 As shown, including: The material matching module 1 is used to match the best teaching material based on the user's history of learning ENT surgery; Model generation module 2, used to generate a virtual reality model based on the best teaching materials; An entry guidance module 3 is used to guide the user to enter the virtual reality model; The interactive control module 4 is used to control the virtual reality model to continuously perform teaching interaction with the user after the user enters the virtual reality model.

[0042] The interaction control module controls the virtual reality model to continuously perform teaching interaction with the user, including: Control the virtual reality model to start teaching the user; Determine multiple target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the multiple target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress; When the virtual reality model teaches the user and enters any target teaching progress, it continuously identifies whether the user has negative emotions; If so, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user are determined to determine the time period and strategy for improving the emotions; When the virtual reality model teaches the user to enter a period of mood improvement, the user's negative mood is improved based on the mood improvement strategy.

[0043] The interactive control module determines the time period and strategy for improving emotions based on the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user, including: If the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user learning the remaining teaching content; Otherwise, the pre-teaching gap of the first remaining teaching content, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content in the remaining teaching content sequence are taken as the opportunity periods for mood improvement; Determine target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; Generate a target timeline of times when mood improves; Content output control mechanism to plan the output content sequence of target improvement knowledge on the target timeline; Determine the mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

[0044] The interactive control module plans a content output control mechanism for outputting a content sequence of target improvement knowledge on a target time axis, including: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies the local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

[0045] The local sequence constraints include: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

[0046] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A virtual reality-based ENT surgery teaching method, characterized in that: include: Match the best teaching materials based on the user's history of learning ENT surgery; Generate virtual reality models based on the best teaching materials; Guide users into the virtual reality model; After the user enters the virtual reality model, the virtual reality model is controlled to continuously perform teaching interaction with the user.

2. The virtual reality-based ENT surgery teaching method according to claim 1, characterized in that: The controlling the virtual reality model to continuously perform teaching interaction with the user includes: Control the virtual reality model to start teaching the user; Determine multiple target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the multiple target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress; When the virtual reality model teaches the user and enters any target teaching progress, it continuously identifies whether the user has negative emotions; If so, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user are determined to determine the time period and strategy for improving the emotions; When the virtual reality model teaches the user to enter a period of mood improvement, the user's negative mood is improved based on the mood improvement strategy.

3. The virtual reality-based ENT surgery teaching method according to claim 2, characterized in that: The remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user, determine the time period for emotion improvement and the emotion improvement strategy, including: If the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user learning the remaining teaching content; Otherwise, the pre-teaching gap of the first remaining teaching content, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content in the remaining teaching content sequence are taken as the opportunity periods for mood improvement; Determine target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; Generate a target timeline of times when mood improves; Content output control mechanism to plan the output content sequence of target improvement knowledge on the target timeline; Determine the mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

4. The virtual reality-based ENT surgery teaching method according to claim 3, characterized in that: The content output control mechanism for planning the output content sequence of target improvement knowledge on the target timeline includes: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies a local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

5. The virtual reality-based ENT surgery teaching method according to claim 4, characterized in that: The local sequence constraints include: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

6. A virtual reality-based ENT surgery teaching system, characterized in that: include: The material matching module is used to match the best teaching material based on the user's history of learning ENT surgery; Model generation module, used to generate virtual reality models based on the best teaching materials; An entry guidance module is used to guide the user to enter the virtual reality model; The interactive control module is used to control the virtual reality model to continuously perform teaching interaction with the user after the user enters the virtual reality model.

7. The virtual reality-based ENT surgery teaching system according to claim 6, characterized in that: The interaction control module controls the virtual reality model to continuously perform teaching interaction with the user, including: Control the virtual reality model to start teaching the user; Determine multiple target teaching progresses from the remaining teaching progress sequence of the virtual reality model teaching the user; wherein the multiple target teaching progresses include: the remaining teaching progress in the remaining teaching progress sequence whose probability is greater than or equal to the probability threshold and the previous remaining teaching progress of the remaining teaching progress whose probability is greater than or equal to the probability threshold; the probability represents the degree to which the user may have negative emotions when the virtual reality model enters the remaining teaching progress; When the virtual reality model teaches the user and enters any target teaching progress, it continuously identifies whether the user has negative emotions; If so, the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user are determined to determine the time period and strategy for improving the emotions; When the virtual reality model teaches the user to enter a period of mood improvement, the user's negative mood is improved based on the mood improvement strategy.

8. The virtual reality-based ENT surgery teaching system according to claim 7, characterized in that: The interactive control module determines the time period and strategy for improving emotions based on the remaining teaching content sequence of the target teaching progress entered by the user through the virtual reality model and the knowledge base for improving the bad emotions of the user, including: If the importance of the first remaining teaching content in the remaining teaching content sequence is greater than or equal to the importance threshold, the previous teaching interval of the first remaining teaching content in the remaining teaching content sequence is used as the mood improvement opportunity period; wherein the importance represents the importance of the user learning the remaining teaching content; Otherwise, the pre-teaching gap of the first remaining teaching content, the teaching period of the first remaining teaching content, and the post-teaching gap of the first remaining teaching content in the remaining teaching content sequence are taken as the opportunity periods for mood improvement; Determine target improvement knowledge from the improvement knowledge base; wherein the target improvement knowledge includes: improvement knowledge in the improvement knowledge base with the smallest time difference between the knowledge output requirement time and the emotion improvement opportunity time period; Generate a target timeline of times when mood improves; Content output control mechanism to plan the output content sequence of target improvement knowledge on the target timeline; Determine the mood improvement strategy based on the target timeline after the output control mechanism planning is completed.

9. The virtual reality-based ENT surgery teaching system according to claim 8, characterized in that: The interactive control module plans a content output control mechanism for outputting a content sequence of target improvement knowledge on a target time axis, including: Determine a plurality of target local sequences from the output content sequence; wherein each target local sequence satisfies a local sequence constraint; Plan content output control mechanisms on the target timeline, including: Traverse each target local sequence in sequence according to the sequence order of each target local sequence in the output content sequence; During each traversal, if the output content in the traversed target partial sequence is unique, based on the content output requirement duration of the output content in the traversed target partial sequence, a first target time interval is set at the front of the front idle area on the target time axis, and a first control strategy is assigned to the first target time interval, including: controlling the output content in the traversed target partial sequence to be output in the first target area in the virtual reality model; wherein the first target area continuously presents a first standard position relationship with the real-time gaze area of ​​the user in the virtual reality model; Otherwise, based on the maximum output requirement duration of the output content in the traversed target local sequence, a second target time interval is set at the front of the idle area on the target time axis, and a second control strategy is assigned to the second target time interval, including: controlling the output content in the traversed target local sequence to be output in the second target area in the virtual reality model; wherein the second target area continues to have a second standard position relationship with the real-time gaze area of ​​the user in the virtual reality model.

10. The virtual reality-based ENT surgery teaching system according to claim 9, characterized in that: The local sequence constraints include: Constraint 1: The same target local sequence contains one output content or at least two consecutive output contents; And constraint 2: in a target partial sequence containing at least two consecutive output contents, each pair of output contents has at least one common standard content association relationship; And constraint three: all output contents in the output content sequence are included in different target local sequences.

Images