Eustachian tube blowing VR training method and system based on image sensor

Abstract

The invention discloses a Eustachian tube blowing VR training method and system based on an image sensor, and relates to the technical field of biological feature recognition, the method comprises the steps that training tasks are set, corresponding VR scenes are constructed, and each task corresponds to a tympanic membrane standard motion model; a user wears VR equipment to select a training task, the system collects tympanic membrane image data, extracts dynamic characteristic parameters such as tympanic membrane displacement, deformation curvature and response frequency, compares the dynamic characteristic parameters with a standard model, generates a training score and displays the training score in a VR environment. The technical problems that the training effect is difficult to guarantee and the user experience is poor due to the fact that an existing Eustachian tube blowing training method lacks an objective quantitative evaluation and real-time feedback mechanism are solved, objective quantitative evaluation and real-time feedback of the tympanic membrane dynamic characteristics are achieved by combining the VR technology and the image sensor technology, and the training efficiency is improved. And the technical effects of training effect and user experience are obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of biometric recognition technology, and in particular to an image sensor-based Eustachian tube inflation VR training method and system. Background Art

[0002] Currently, Eustachian tube stretching training primarily relies on traditional clinical methods, typically performed through manual observation and patient self-manipulation. These methods have limitations, such as a lack of precise training evaluation criteria, difficulty in real-time monitoring of user performance, and reliance on subjective judgment by physicians for feedback, which can lead to unstable training outcomes and individual differences. Furthermore, these methods often lack interactivity and personalized adjustments, and cannot dynamically adjust training difficulty based on the user's real-time performance, making it difficult to fully guarantee training efficiency and effectiveness. Summary of the Invention

[0003] This application provides an image sensor-based Eustachian tube blowing VR training method and system, which is used to solve the technical problems that the existing Eustachian tube blowing training methods lack objective quantitative evaluation and real-time feedback mechanism, resulting in difficulty in ensuring training effects and poor user experience.

[0004] A first aspect of the present application provides an image sensor-based Eustachian tube blowing VR training method, the method comprising: setting a Eustachian tube blowing training task and constructing a Eustachian tube blowing training VR scene, wherein each Eustachian tube blowing training task includes a corresponding standard tympanic membrane motion model; a user selects a Eustachian tube blowing training task in the current Eustachian tube blowing training VR scene by wearing a VR virtual device, and displays the corresponding matching standard tympanic membrane motion model; collecting tympanic membrane image data corresponding to the user under the Eustachian tube blowing training task according to the image sensor; extracting tympanic membrane dynamic feature parameters of the tympanic membrane image data, including tympanic membrane displacement, deformation curvature and response frequency; comparing the tympanic membrane dynamic feature parameters with the matching standard tympanic membrane motion model, outputting a training score result, and displaying the training score result through VR interactive display.

[0005] According to a second aspect of the present application, a Eustachian tube insufflation VR training system based on an image sensor is provided, the system comprising: a training scene construction module, the training scene construction module being used to set Eustachian tube insufflation training tasks and construct a Eustachian tube insufflation training VR scene, wherein each Eustachian tube insufflation training task includes a corresponding standard tympanic membrane motion model; a training task matching module, the training task matching module being used to enable a user to select a Eustachian tube insufflation training task in a current Eustachian tube insufflation training VR scene by wearing a VR virtual device, and to display the corresponding matching standard tympanic membrane motion model; a tympanic membrane image data acquisition module, the tympanic membrane image data acquisition module being used to acquire tympanic membrane image data corresponding to the user under the Eustachian tube insufflation training task according to an image sensor; a dynamic feature parameter extraction module, the dynamic feature parameter extraction module being used to extract tympanic membrane dynamic feature parameters of the tympanic membrane image data, including tympanic membrane displacement, deformation curvature, and response frequency; and a training score interaction module, the training score interaction module being used to compare the tympanic membrane dynamic feature parameters with the matching standard tympanic membrane motion model, output a training score result, and perform VR interactive display on the training score result.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0007] The image sensor-based Eustachian tube blowing VR training method and system provided in the present application relate to the field of biometric recognition technology. By setting training tasks and constructing corresponding VR scenes, the user's tympanic membrane image data is collected, and dynamic features such as tympanic membrane displacement, deformation curvature and response frequency are extracted. After comparison with the standard model, a training score is generated and interactively displayed in VR form, thereby achieving accurate evaluation and feedback of the training process. This solves the technical problem that the existing Eustachian tube blowing training method lacks objective quantitative evaluation and real-time feedback mechanism, resulting in difficulty in ensuring training effects and poor user experience. By combining VR technology and image sensor technology, objective quantitative evaluation and real-time feedback of the dynamic features of the tympanic membrane are achieved, significantly improving the training effect and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A flowchart of the image sensor-based VR training method for Eustachian tube expansion provided in an embodiment of the present application;

[0010] Figure 2Schematic diagram of the structure of the Eustachian tube expansion VR training system based on image sensor provided in an embodiment of the present application.

[0011] Explanation of the accompanying drawings: training scene construction module 11, training task matching module 12, tympanic membrane image data acquisition module 13, dynamic feature parameter extraction module 14, training score interaction module 15. DETAILED DESCRIPTION

[0012] This application provides an image sensor-based Eustachian tube blowing VR training method and system, which is used to solve the technical problems that the existing Eustachian tube blowing training methods lack objective quantitative evaluation and real-time feedback mechanism, resulting in difficulty in ensuring training effects and poor user experience.

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0014] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] Example 1, as Figure 1 As shown, the present application provides a Eustachian tube expansion VR training method based on an image sensor, the method comprising:

[0016] P10: Set up Eustachian tube stretching training tasks and build a Eustachian tube stretching training VR scene, where each Eustachian tube stretching training task includes a corresponding standard tympanic membrane motion model.

[0017] Furthermore, step P10 in the embodiment of the present application further includes:

[0018] P11: Scan the healthy Eustachian tube anatomical structure and establish three-dimensional point cloud data. The Eustachian tube anatomical structure includes the nasal cavity, nasopharynx, Eustachian tube, middle ear, external auditory canal and tympanic membrane in the ear and nasopharynx area; P12: Import the three-dimensional point cloud data into the three-dimensional model and output the Eustachian tube anatomical three-dimensional model; P13: Obtain various Eustachian tube blowing training tasks, and obtain various motion parameter libraries for the various Eustachian tube blowing training tasks; P14: Import various motion parameter libraries into the Eustachian tube anatomical three-dimensional model to simulate the dynamic changes of the tympanic membrane, output various standard motion models of the tympanic membrane, and integrate them to obtain a Eustachian tube blowing training VR scene.

[0019] It should be understood that in this embodiment, it is first necessary to set the Eustachian tube blowing training task and construct a virtual training environment that conforms to the physiological structure and functional requirements of the human body. Through precise modeling and dynamic parameter simulation, it is ensured that the user can obtain training tasks and feedback that are consistent with actual physiological conditions when performing Eustachian tube blowing training.

[0020] In order to build an accurate VR scene for Eustachian tube stretching training and set corresponding training tasks, it is first necessary to scan the anatomical structure of a healthy Eustachian tube and use medical imaging technology (such as CT scan or MRI) to obtain detailed data on the Eustachian tube and its surrounding anatomical areas. These data are converted into three-dimensional point cloud data through high-precision scanning technology. The point cloud data accurately reflects the geometry of the Eustachian tube by sampling multiple points in the target area. This three-dimensional point cloud data not only includes anatomical structures such as the nasal cavity, nasopharynx, Eustachian tube, middle ear, external auditory canal and eardrum, but also captures the relative positions and sizes of these structures, providing basic data for the subsequent construction of an accurate three-dimensional model.

[0021] The acquired 3D point cloud data is then imported into professional 3D modeling software. Using the software's modeling algorithms and tools, the point cloud data is processed and fitted to generate a 3D model of the Eustachian tube anatomy. This model intuitively displays the morphology and spatial relationships of the Eustachian tube and its surrounding structures in 3D, providing an accurate anatomical foundation for subsequent dynamic simulations. During the modeling process, ensuring the model's accuracy and detail ensures that it accurately reflects the anatomical structure.

[0022] After completing the construction of the anatomical model, various Eustachian tube stretching training tasks were further acquired, and for each training task, a corresponding motion parameter library was extracted. This motion parameter library contains the dynamic characteristic parameters of the tympanic membrane under different training tasks, such as tympanic membrane displacement, deformation curvature, response frequency, etc. These parameters are based on a large amount of experimental data and clinical research, and can accurately reflect the movement characteristics of the tympanic membrane under normal Eustachian tube function, providing a standard basis for subsequent simulation and evaluation. The motion parameter library was established by collecting and analyzing tympanic membrane movement data under different training conditions and organizing it into a parameter set that can be simulated and compared.

[0023] Finally, each motion parameter library is imported into the previously constructed 3D model of the Eustachian tube anatomy. Using computer simulation technology, the dynamic changes of the tympanic membrane are simulated based on the imported motion parameters. This process can realistically reproduce the movement of the tympanic membrane under different training tasks and generate various standard tympanic membrane motion models. These standard motion models will be integrated into the Eustachian tube expansion training VR scene, providing users with intuitive training goals and references. When training in the virtual environment, users can clearly see the difference between their own tympanic membrane movement and the standard model, allowing them to better adjust their training methods and improve training results.

[0024] Through the above steps, this application not only constructs a highly realistic Eustachian tube blowing training VR scene, but also provides an accurate standard tympanic membrane movement model for each training task, greatly improving the scientificity and effectiveness of the training.

[0025] Furthermore, step P13 of the embodiment of the present application further includes:

[0026] P13-1: Eustachian tube blowing action samples for each Eustachian tube blowing training task; P13-2: Collecting corresponding tympanic membrane image sample data for multiple user groups under the Eustachian tube blowing action samples; P13-3: Extracting tympanic membrane dynamic feature samples from the tympanic membrane image sample data, including preset reference displacement range samples, deformation curvature threshold samples, and response frequency interval samples; P13-4: Generate various motion parameter libraries according to the tympanic membrane dynamic feature samples corresponding to each Eustachian tube blowing training task.

[0027] Optionally, in order to generate a motion parameter library for each Eustachian tube stretching training task, a large amount of real data and samples can be collected to ensure that each training task can accurately simulate and evaluate the user's actual training effect.

[0028] First, for each Eustachian tube stretching training task, corresponding Eustachian tube stretching motion samples were collected. These motion samples were designed based on standardized training movements and covered stretching movements of varying difficulty levels and training objectives. These motion samples served as the basis for the subsequent collection of tympanic membrane image sample data, ensuring that the collected data accurately reflected the dynamic characteristics of the tympanic membrane under different training tasks.

[0029] Next, we collected sample data of eardrum images from multiple user groups performing the aforementioned Eustachian tube stretching maneuvers. This selection took into account factors such as age, gender, and health status to ensure diverse and representative sample data. Image sensors were used to capture eardrum images of users performing the Eustachian tube stretching maneuver. These image samples served as the basis for subsequent feature extraction and parameter generation.

[0030] Furthermore, the image data is used to extract dynamic feature samples of the eardrum. These features include key parameters such as the displacement range, deformation curvature and response frequency of the eardrum. In this step, the collected eardrum images need to be processed first, and the position and morphological changes of the eardrum in each frame of the image are extracted, and then the displacement range of the eardrum at each moment during the expansion process is calculated. The deformation curvature refers to the degree of deformation of the eardrum during the expansion process, which is usually obtained by performing a curvature analysis on the edge of the eardrum in the image. The response frequency refers to the frequency response of the eardrum under a specific expansion action, which reflects the reaction rate of the eardrum to external stimuli. In order to ensure the scientific nature of the training process, all these feature data must be compared with the preset benchmark samples, and the deformation curvature threshold and response frequency range must be set according to clinical experience to ensure the accuracy and operability of the data.

[0031] Finally, the extracted tympanic membrane dynamic feature samples are used to generate a motion parameter library for each Eustachian tube stretching training task. This motion parameter library is a collection of multiple tympanic membrane dynamic feature samples. It not only includes the tympanic membrane's displacement range, deformation curvature, and response frequency, but also includes task-specific parameters such as target training intensity, movement duration, and frequency. This motion parameter library serves as the core data for each training task. By comparing and simulating the user's actual performance, the system helps accurately evaluate training results and provide real-time feedback on training scores.

[0032] P20: The user wears the VR virtual device and selects the Eustachian tube stretching training task in the current Eustachian tube stretching training VR scene, and the corresponding matching tympanic membrane standard movement model is displayed.

[0033] Specifically, users wear VR headsets to enter a virtual environment for Eustachian tube stretching training and select a specific training task within that environment. The core goal of this step is to allow users to interactively enter the virtual training scene and select specific training tasks based on their needs, thereby achieving their training goals within the virtual environment.

[0034] First, the user dons a VR headset and controller to enter a fully immersive virtual scene. This scene is based on the Eustachian tube stretching training environment constructed in the previous steps. Using high-precision graphics rendering technology, it realistically recreates the anatomy of the otolaryngology and the training tasks. This virtual environment not only faithfully reproduces the physiological action of Eustachian tube stretching but also displays the dynamic changes of the eardrum, allowing the user to clearly observe how the eardrum responds to different stretching movements during training.

[0035] In the virtual environment, users are presented with a variety of training tasks, each corresponding to a different target movement and training intensity. Using the VR device's controller or gesture recognition system, users select the Eustachian tube stretching task they wish to perform. These tasks can vary in movement amplitude, stretch duration, and other factors, allowing users to flexibly select tasks based on their training needs.

[0036] Once the user selects a training task, the system automatically displays a standard tympanic membrane motion model that matches the task. This standard motion model was generated in the previous step based on the tympanic membrane's dynamic characteristics and motion parameter library. It displays parameters such as the standard displacement, deformation, and response frequency that the tympanic membrane should have under this task. In the virtual environment, the user can observe the dynamic changes of the standard motion model in real time and understand the differences between themselves and the standard model through visual feedback. This process helps users understand the training goals in an intuitive way and then adjust their movements to achieve the best training results.

[0037] Through this virtual training method, users can not only perform Eustachian tube blowing training in a highly interactive and dynamic feedback environment, but also use virtual reality technology to accurately simulate various blowing movements, thereby improving the scientific nature and targetedness of the training.

[0038] P30: Using an image sensor, collect tympanic membrane image data corresponding to the user during the Eustachian tube expansion training task.

[0039] Optionally, after the user selects and begins the Eustachian tube stretching training task, an image sensor is activated to capture image data of the user's eardrum during training. The image sensor is a key component of this system, capturing the dynamic changes of the user's eardrum in real time during the Eustachian tube stretching exercise. To ensure that the captured image data accurately reflects the movement characteristics of the eardrum, the image sensor must possess high resolution, high frame rate, and low noise to meet medical-grade accuracy requirements.

[0040] The image sensor is typically mounted within the user's field of view, or aimed directly at the ear, nose, and pharynx, capturing high-resolution image data of the eardrum. This data captures how the eardrum deforms, moves, and responds to the Eustachian tube's expansion during training.

[0041] The core of the image acquisition process is to ensure that sufficiently detailed and high-frequency images of the eardrum are captured for subsequent dynamic feature analysis. Specifically, the system needs to record every change in the eardrum in real time during training, capturing its displacement, deformation, and any subtle movements. To ensure data accuracy and integrity, the image sensor needs to operate continuously throughout the entire training task and be able to stably capture changes in the eardrum under varying training intensities, time periods, and user movements.

[0042] These tympanic membrane image data serve as the foundation for subsequent training analysis. They are passed as input data to the subsequent image processing and feature extraction modules for further processing and analysis. The precise acquisition of these image data provides the system with visual data on the dynamic changes in the tympanic membrane, providing a reliable basis for subsequent training effect evaluation and feedback. Therefore, the high precision and stability of the image sensor are key to ensuring the successful execution of this step. This process not only ensures the accuracy and integrity of the data, but also improves the efficiency and user experience of the entire training system through automated and real-time acquisition.

[0043] P40: Extracting eardrum dynamic characteristic parameters of the eardrum image data, including eardrum displacement, deformation curvature, and response frequency.

[0044] Furthermore, step P40 in this embodiment of the present application further includes:

[0045] P41: The image sensor is installed in one end of the ear canal wearing device close to the internal auditory canal opening, and the ear canal wearing device is connected to the VR virtual device through a wireless communication module; P42: The eardrum image data is transmitted to the data processing terminal of the VR virtual device, and the eardrum dynamic feature parameters of the eardrum image data are extracted at the data processing terminal.

[0046] It should be understood that to accurately assess the dynamic changes in the tympanic membrane during Eustachian tube expansion training, it is necessary to extract key dynamic characteristic parameters from the collected tympanic membrane image data, including tympanic membrane displacement, deformation curvature, and response frequency. These parameters can comprehensively reflect the movement and functional performance of the tympanic membrane during training, providing an important basis for subsequent evaluation of training results.

[0047] During the specific implementation process, the image sensor is built into the end of the ear canal-worn device near the opening of the internal auditory canal. This design ensures that the image sensor can capture the dynamic changes of the eardrum at close range and with high precision, while avoiding interference from external ambient light, thereby improving the quality and stability of image acquisition. The ear canal-worn device is connected to the VR virtual device through a wireless communication module to achieve real-time transmission of image data. The wireless communication module can use Bluetooth, Wi-Fi or other low-latency, high-bandwidth wireless communication technologies to ensure that image data can be transmitted quickly and stably to the data processing terminal of the VR device.

[0048] After the collected tympanic membrane image data is transmitted to the data processing terminal of the VR virtual device through the wireless communication module, the system uses professional image processing algorithms to analyze these data and extract the dynamic characteristic parameters of the tympanic membrane. For example, first, the collected tympanic membrane image data is preprocessed, including denoising, contrast enhancement and other operations to improve the clarity and usability of the image. This preprocessing step can effectively remove noise interference in the image, while enhancing the contrast between the tympanic membrane and the surrounding tissue, providing a clearer image basis for subsequent feature extraction.

[0049] Afterwards, the system begins to extract the dynamic characteristic parameters of the eardrum, including the displacement of the eardrum, deformation curvature and response frequency. Among them, the displacement of the eardrum is calculated by analyzing the position changes of the eardrum at different time points. This parameter can reflect the movement amplitude of the eardrum during the expansion of the Eustachian tube. The deformation curvature is calculated by analyzing the changes in the curvature of the eardrum surface. This parameter can reflect the degree of deformation of the eardrum during the expansion process and help evaluate the elastic properties of the eardrum. The response frequency can be calculated by analyzing the response of the eardrum to sound stimuli of different frequencies. This parameter can reflect the vibration characteristics of the eardrum at different frequencies and help evaluate the functional status of the eardrum.

[0050] Through the above steps, the dynamic characteristic parameters of the eardrum can be accurately extracted and transmitted to the subsequent comparison module for comparison and analysis with the standard motion model. This process not only improves the scientific nature and effectiveness of training, but also helps users better adjust training movements through real-time feedback, enhancing the training experience.

[0051] P50: Compare the tympanic membrane dynamic feature parameters with the matching tympanic membrane standard motion model, output a training score result, and display the training score result in VR interaction.

[0052] Furthermore, step P50 in the embodiment of the present application further includes:

[0053] P51: The matching eardrum standard motion model includes a preset reference displacement range, a deformation curvature threshold, and a response frequency interval; P52: The eardrum displacement, deformation curvature, and response frequency of the eardrum dynamic characteristic parameters are compared with the preset reference displacement range, deformation curvature threshold, and response frequency interval, and the displacement matching degree, response timing deviation, and frequency similarity are output; P53: The displacement matching degree, response timing deviation, and frequency similarity are weightedly calculated, and the training score result is output.

[0054] Optionally, after the user completes the Eustachian tube blowing training task, the user's performance needs to be evaluated. To this end, the dynamic characteristic parameters of the tympanic membrane generated by the user during the training process (including tympanic membrane displacement, deformation curvature and response frequency) are compared with the corresponding matching tympanic membrane standard motion model. The matching tympanic membrane standard motion model is established based on a large amount of data from healthy people, and includes key parameters such as a preset baseline displacement range, deformation curvature threshold and response frequency range. By comparing the user's actual tympanic membrane dynamic characteristic parameters with the parameter range in the standard model, the system can quantitatively evaluate the user's training performance and output the training score results. The score results will be intuitively displayed to the user through the VR interactive interface, so that the user can clearly understand his or her performance in training and the differences between him or her and the standard model.

[0055] The matching standard motion model of the eardrum includes a preset reference displacement range, a deformation curvature threshold, and a response frequency interval. These parameters are derived by analyzing the dynamic characteristics of the eardrum of healthy people during the Eustachian tube expansion process, and can reflect the movement characteristics of the eardrum under normal functional conditions. The preset reference displacement range defines the displacement amplitude of the eardrum during the expansion process; the deformation curvature threshold sets the maximum allowable degree of eardrum deformation; and the response frequency interval specifies the response range of the eardrum to sound stimuli of different frequencies. Together, these parameters constitute a standard model for evaluating user training performance.

[0056] In the specific implementation process, the eardrum displacement in the user's eardrum dynamic characteristic parameters is first compared with the preset reference displacement range to determine whether the user's eardrum displacement is within the normal range. If the user's eardrum displacement exceeds or falls below the preset range, it indicates that there is a deviation in its displacement performance in the training task. Next, the user's eardrum deformation curvature is compared with the deformation curvature threshold to evaluate whether the degree of deformation of the eardrum during the blowing process meets the normal standard. If the deformation curvature exceeds the threshold, it may mean that there is a problem with the elasticity or function of the eardrum. Finally, the user's eardrum response frequency is compared with the response frequency range to determine whether the response characteristics of the eardrum to sound stimuli of different frequencies are consistent with the standard model. Through these comparison operations, the system can output three key evaluation indicators: displacement matching degree, response timing deviation and frequency similarity.

[0057] The three evaluation metrics are then weighted to produce the final training score. During this weighted calculation, the weight of each metric can be flexibly adjusted based on its importance in training. For example, if displacement matching is more critical in the training task, it can be given a higher weight; if response frequency accuracy is more important, the weight of frequency similarity can be increased accordingly. In this way, the system comprehensively considers multiple aspects of performance, ensuring that the final score fully and accurately reflects the user's training performance.

[0058] Finally, the training score results are intuitively displayed to the user through a VR interactive interface. In the VR scene, users can not only see their scores, but also clearly understand the specific differences between their eardrum displacement, deformation curvature, and response frequency compared to the standard model through color coding, charts, or animations. This intuitive display method not only allows users to quickly understand their training performance, but also helps them identify areas for improvement, thereby better adjusting training strategies and improving training results.

[0059] Furthermore, step P52 of the embodiment of the present application further includes:

[0060] P52-1: The eardrum displacement, deformation curvature and response frequency of the eardrum dynamic characteristic parameters are compared with the preset reference displacement range, deformation curvature threshold and response frequency interval respectively; P52-2: The Euclidean distance of the eardrum displacement is calculated within the preset reference displacement range, and the displacement matching degree is output; P52-3: The DTW dynamic warping algorithm is performed on the deformation curvature within the deformation curvature threshold, and the response timing deviation is output; P52-4: The response frequency is Fourier transformed and cosine similarity analysis is performed on the response frequency interval, and the frequency similarity is output.

[0061] In one possible embodiment of the present application, a detailed description is provided of how to accurately compare the dynamic characteristic parameters of the eardrum with the key parameters in the preset standard model, and to evaluate the degree of match of each characteristic parameter using various mathematical analysis methods. The purpose of this process is to provide accurate training feedback through detailed quantification of the user's training tasks, and to provide a basis for subsequent score calculation.

[0062] Specifically, three key indicators of the tympanic membrane's dynamic characteristic parameters—tympanic membrane displacement, deformation curvature, and response frequency—are compared against a preset baseline displacement range, deformation curvature threshold, and response frequency range. The difference between each characteristic parameter and the standard model affects subsequent training scores, so each comparison must be performed accurately. The system uses these comparisons to evaluate the user's actual performance and determine whether it meets the requirements of the standard model.

[0063] First, the Euclidean distance is calculated to measure the difference between the user's eardrum displacement and the standard baseline displacement range. Euclidean distance is a common metric used to calculate the straight-line distance between two points. In this application, the difference between the actual eardrum displacement and the standard displacement range in multidimensional space is quantified using Euclidean distance. A smaller distance value indicates that the user's eardrum displacement is closer to the standard model; conversely, a smaller distance value indicates a significant deviation from the standard model. The calculated degree of displacement matching provides a quantitative basis for subsequent scoring.

[0064] For the comparison of deformation curvature, the dynamic time warping (DTW) algorithm can be used to analyze the deformation curvature. The DTW algorithm is a technique for measuring the similarity of two time series. It can compare data at different time scales to obtain the optimal alignment of the two series. Here, the deformation curvature data is dynamically warped and analyzed using the DTW algorithm to calculate the response timing deviation. This analysis can not only measure the degree of deformation of the eardrum, but also handle the timing deviation caused by speed changes during training. The DTW algorithm can effectively compare the dynamic changes of deformation curvature and output a numerical value to evaluate the timing deviation, which provides an important timing analysis dimension for training evaluation.

[0065] In the comparison of response frequencies, the system first performs a Fourier transform on the response frequency, converting the signal in the frequency domain from the time domain to the frequency domain signal. The Fourier transform can reveal the intensity of different frequency components in the signal, and makes the analysis of frequency characteristics more intuitive and accurate. Then, the system performs a cosine similarity analysis on the converted frequency signal and the preset response frequency interval. Cosine similarity is a commonly used indicator to measure the similarity between two vectors. The value range is -1 to 1. The closer the value is to 1, the more similar the frequency characteristics of the two signals are. In this step, the system calculates the similarity between the user's response frequency and the standard frequency interval to obtain the frequency similarity. This result reflects the degree of match between the frequency of the eardrum's response to the blowing action and the standard model.

[0066] This mathematical algorithm-based comparison method ensures the scientificity, objectivity and accuracy of training evaluation, and can provide users with personalized, data-driven training feedback.

[0067] Furthermore, step P50 in the embodiment of the present application further includes:

[0068] P51a: The ear canal wearing device also includes a pressure sensor, which is installed in one end of the ear canal wearing device close to the external auditory canal opening; P52a: The blowing airflow data corresponding to the user in the Eustachian tube blowing training task is collected according to the pressure sensor; P53a: The blowing airflow data is transmitted to the data processing terminal of the VR virtual device, and the airflow characteristic parameters of the blowing airflow data are extracted at the data processing terminal, and the training score result is outputted by auxiliary calculation according to the airflow characteristic parameters.

[0069] Optionally, in order to more comprehensively evaluate the user's performance in Eustachian tube blowing training, blowing airflow data can also be introduced as an auxiliary evaluation indicator. Specifically, in addition to the built-in image sensor, the ear canal-worn device is also equipped with a pressure sensor. The pressure sensor is built into the end of the ear canal-worn device near the external auditory canal opening and is used to collect the blowing airflow data generated by the user during the Eustachian tube blowing training task.

[0070] When the user begins Eustachian tube blowing training, the pressure sensor will monitor and record the changes in airflow pressure generated by the user's blowing process in real time. These airflow data are transmitted to the data processing terminal of the VR virtual device through the wireless communication module of the ear canal wearable device. At the data processing terminal, the system processes and analyzes the collected blowing airflow data and extracts the airflow characteristic parameters. These airflow characteristic parameters may include the peak pressure, duration, pressure change rate, etc. of the airflow. These parameters can reflect the user's force and breathing control ability during the blowing process.

[0071] The extracted airflow characteristic parameters are compared with the preset standard airflow characteristic range to assess whether the user's blowing airflow meets training requirements. For example, if the user's airflow pressure peak is too high or too low, or the airflow duration is too short, it may indicate that the user is using improper force or poor breathing control during the blowing process. Through comparative analysis, the system can output airflow-related evaluation indicators, such as airflow matching degree and airflow stability.

[0072] Finally, the evaluation results of the airflow characteristic parameters are combined with the evaluation results of the tympanic membrane dynamic characteristic parameters to comprehensively calculate the user's training score. The evaluation results of the airflow characteristic parameters can be used as a weight factor to assist the evaluation of the tympanic membrane dynamic characteristic parameters to more comprehensively reflect the user's training performance. For example, if the user's tympanic membrane dynamic characteristics perform well, but the airflow characteristic parameters show improper force, the training score may be affected to a certain extent. By introducing airflow data as an auxiliary evaluation indicator, the system can more accurately evaluate the user's training effect, provide users with more comprehensive feedback, and help users better adjust their training strategies and improve training results.

[0073] Furthermore, step P53a of the embodiment of the present application further includes:

[0074] P53-1a: Establishing a time series mapping relationship between the tympanic membrane dynamic characteristic parameters and the airflow characteristic parameters; P53-2a: Calculating a response delay index according to the time series mapping relationship, and updating the training score result according to the response delay index.

[0075] It should be understood that by establishing a temporal mapping relationship between the dynamic characteristic parameters of the eardrum and the characteristic parameters of the airflow, the calculation of training scores and user training feedback can be further optimized. This process can accurately analyze the temporal relationship between the user's blowing and stretching movements and the eardrum's response, providing users with more accurate training assessments, especially analyzing the coordination of blowing and stretching movements.

[0076] First, a time-series mapping relationship is established between the dynamic characteristic parameters of the eardrum and the characteristic parameters of the airflow. In this process, the system aligns the dynamic characteristics of the eardrum, such as displacement, deformation curvature and response frequency, with the airflow characteristics (such as pressure and fluctuation of the airflow) to reflect the relationship between the change in airflow and the eardrum response when the blowing action occurs. The establishment of this time-series mapping relationship is based on the matching of the time points of each feature, that is, matching the dynamic changes of the eardrum and the pressure changes of the airflow with the specific time series, so as to accurately analyze the response of the eardrum to the changes in airflow within a specific time.

[0077] Establishing this temporal mapping is crucial because the displacement and deformation of the eardrum are triggered by external airflow stimuli, often with a time delay. This precise mapping quantifies the time difference between airflow and eardrum response, providing data support for subsequent response delay analysis.

[0078] Next, the response delay index is calculated based on the established timing mapping relationship. Response delay refers to the time difference between the user's blowing action (such as airflow output) and the physical response of the eardrum. This delay reflects the user's coordination of the blowing action during training. A shorter response delay usually means that the user can perform the blowing action quickly and accurately, and make the eardrum respond in time. A longer response delay may indicate that the user's action execution is not coordinated enough, or there is a problem with the coordination of the action and airflow.

[0079] This response delay indicator can be calculated by comparing the time series of airflow data and eardrum response data. The size of the response delay not only reflects the coordination of the user's blowing and stretching movements during training, but also provides an additional evaluation dimension for the system. Based on this delay indicator, the system can further optimize the training score and evaluate the training effect by determining the length of the delay. If the delay is long, it means that there is a certain coordination problem between the blowing and stretching movement and the eardrum response. The system may suggest that the user adjust the rhythm of the movement or strengthen certain specific training tasks.

[0080] Finally, the training score is updated based on this metric. If the response delay falls within the expected range, the user will receive a higher score, indicating better coordination of the blowing and stretching movements. If the delay is greater, the system can adjust the score and provide feedback to help the user identify areas of the movement that need improvement.

[0081] This process enables the system to more comprehensively analyze the user's training performance, particularly the coordination and precision of movements during training, thereby providing more targeted and personalized feedback. This feedback mechanism can help users quickly identify problems during blowing and stretching training, optimize movement execution, and ultimately improve training results.

[0082] Furthermore, the embodiment of the present application further includes step P60, which further includes:

[0083] P61: The Eustachian tube blowing training task is managed in multiple levels through the permission module, wherein the training difficulty of each level of Eustachian tube blowing training task increases successively; P62: Recording the training score data of the user; P63: If the mean of the training score data of the user is greater than the preset score mean, unlocking the training permission for the next level of Eustachian tube blowing training task.

[0084] Specifically, in this embodiment of the present application, step P60 is further introduced to systematically manage the Eustachian tube stretching training task and track the user's training progress. Specifically, the Eustachian tube stretching training task is managed at multiple levels through a permissions module, with the training difficulty of each level increasing. This hierarchical design helps users gradually adapt to the training requirements, starting with simple tasks and gradually transitioning to more challenging ones, ensuring that users have sufficient capabilities and skills before moving on to higher-level tasks.

[0085] The system records the user's scores for each training task. These scores not only reflect the user's performance in the current task but also provide an important basis for subsequent training progress evaluation. Each time a user completes a training task, the system automatically stores the score in the user's training profile for subsequent analysis. This allows the system to continuously monitor the user's training performance and provide personalized feedback and suggestions.

[0086] When the average of a user's training scores exceeds a preset average, the system automatically unlocks the next level of training tasks. The average score reflects the user's overall performance across multiple training tasks and serves as a metric for assessing their ability to challenge more challenging tasks. If a user's training scores reach or exceed the preset average, the system will allow them to advance to a more difficult training task, ensuring they train within their capabilities while preventing overly easy or overly difficult tasks from impacting their training effectiveness.

[0087] Through this multi-level management and dynamic unlocking mechanism, the system can flexibly adjust training difficulty based on the user's actual performance, providing a personalized training experience. Users can maintain a challenging level through gradually increasing task difficulty, while ensuring that each stage of training effectively improves their skills. This mechanism not only helps increase user motivation but also ensures continuity and stability in training, resulting in better learning outcomes.

[0088] In summary, the embodiments of the present application have at least the following technical effects:

[0089] This application sets a training task that includes a standard tympanic membrane motion model and constructs a corresponding VR scene. After the user wears the VR device and selects the task, the system collects tympanic membrane image data through the image sensor, extracts dynamic feature parameters such as tympanic membrane displacement, deformation curvature and response frequency, and compares them with the standard model. The training score results are output and VR interactive feedback is provided, thereby realizing real-time evaluation of training effects and personalized guidance.

[0090] By combining VR technology and image sensor technology, the objective quantitative evaluation and real-time feedback of the dynamic characteristics of the tympanic membrane are achieved, significantly improving the training effect and user experience.

[0091] Example 2, based on the same inventive concept as the image sensor-based VR training method for the Eustachian tube blowing in the previous embodiment, Figure 2 As shown, the present application provides a VR training system for Eustachian tube expansion based on an image sensor. The system and method embodiments in the present application are based on the same inventive concept. The system includes:

[0092] The training scene construction module 11 is used to set a Eustachian tube blowing training task and construct a Eustachian tube blowing training VR scene, wherein each Eustachian tube blowing training task includes a corresponding standard tympanic membrane movement model.

[0093] The training task matching module 12 is used for the user to select the Eustachian tube blowing training task in the current Eustachian tube blowing training VR scene by wearing a VR virtual device, and display the corresponding matching tympanic membrane standard movement model.

[0094] The tympanic membrane image data acquisition module 13 is configured to acquire tympanic membrane image data corresponding to the user during the Eustachian tube expansion training task using an image sensor.

[0095] The dynamic feature parameter extraction module 14 is used to extract the dynamic feature parameters of the eardrum from the eardrum image data, including eardrum displacement, deformation curvature and response frequency.

[0096] The training score interaction module 15 is used to compare the tympanic membrane dynamic feature parameters with the matching tympanic membrane standard motion model, output the training score result, and perform VR interactive display on the training score result.

[0097] Furthermore, the training scenario construction module 11 is further configured to perform the following steps:

[0098] The healthy Eustachian tube anatomical structure is scanned to establish three-dimensional point cloud data, wherein the Eustachian tube anatomical structure includes the nasal cavity, nasopharynx, Eustachian tube, middle ear, external auditory canal and tympanic membrane in the otorhinolaryngology area; the three-dimensional point cloud data is imported into a three-dimensional model to output a three-dimensional model of the Eustachian tube anatomy; various Eustachian tube blowing training tasks are obtained, and various motion parameter libraries are obtained for each Eustachian tube blowing training task; various motion parameter libraries are respectively imported into the three-dimensional model of the Eustachian tube anatomy to simulate the dynamic changes of the tympanic membrane, and various standard motion models of the tympanic membrane are output, and integrated to obtain a Eustachian tube blowing training VR scene.

[0099] Furthermore, the training scenario construction module 11 is further configured to perform the following steps:

[0100] Eustachian tube blowing action samples for each Eustachian tube blowing training task; collecting corresponding tympanic membrane image sample data under the Eustachian tube blowing action samples for multiple user groups; extracting tympanic membrane dynamic feature samples from the tympanic membrane image sample data, including preset reference displacement range samples, deformation curvature threshold samples, and response frequency interval samples; and generating various motion parameter libraries according to the tympanic membrane dynamic feature samples corresponding to each Eustachian tube blowing training task.

[0101] Furthermore, the dynamic feature parameter extraction module 14 is further configured to perform the following steps:

[0102] The image sensor is installed in one end of the ear canal wearing device close to the internal auditory canal opening, and the ear canal wearing device is connected to the VR virtual device through a wireless communication module; the eardrum image data is transmitted to the data processing terminal of the VR virtual device, and the eardrum dynamic feature parameters of the eardrum image data are extracted at the data processing terminal.

[0103] Furthermore, the training score interaction module 15 is further configured to perform the following steps:

[0104] Among them, the matching eardrum standard motion model includes a preset reference displacement range, deformation curvature threshold and response frequency interval; the eardrum displacement, deformation curvature and response frequency of the eardrum dynamic characteristic parameters are compared with the preset reference displacement range, deformation curvature threshold and response frequency interval respectively, and the displacement matching degree, response timing deviation and frequency similarity are output; the displacement matching degree, response timing deviation and frequency similarity are weightedly calculated, and the training score result is output.

[0105] Furthermore, the training score interaction module 15 is further configured to perform the following steps:

[0106] The ear canal wearing device also includes a pressure sensor, which is arranged in one end of the ear canal wearing device close to the external auditory canal opening; the blowing airflow data corresponding to the user in the Eustachian tube blowing training task is collected according to the pressure sensor; the blowing airflow data is transmitted to the data processing terminal of the VR virtual device, the airflow characteristic parameters of the blowing airflow data are extracted at the data processing terminal, and the training score result is assisted in calculation and output according to the airflow characteristic parameters.

[0107] Furthermore, the training score interaction module 15 is further configured to perform the following steps:

[0108] A time series mapping relationship between the tympanic membrane dynamic characteristic parameter and the airflow characteristic parameter is established; a response delay index is calculated according to the time series mapping relationship, and a training score result is updated according to the response delay index.

[0109] Furthermore, the training score interaction module 15 is further configured to perform the following steps:

[0110] The eardrum displacement, deformation curvature and response frequency of the eardrum dynamic characteristic parameters are compared with the preset reference displacement range, deformation curvature threshold and response frequency interval respectively; wherein, the Euclidean distance calculation is performed on the eardrum displacement within the preset reference displacement range, and the displacement matching degree is output; the DT W dynamic warping algorithm analysis is performed on the deformation curvature within the deformation curvature threshold, and the response timing deviation is output; the response frequency is Fourier transformed and cosine similarity analysis is performed on the response frequency interval, and the frequency similarity is output.

[0111] Furthermore, the system also includes a multi-level management module for performing the following steps:

[0112] The Eustachian tube blowing training task is managed in multiple levels through a permission module, wherein the training difficulty of each level of the Eustachian tube blowing training task increases successively; the training score data of the user is recorded; if the average of the user's training score data is greater than the preset score average, the training permission for the next level of the Eustachian tube blowing training task is unlocked.

[0113] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0115] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. The Eustachian tube stretching VR training method based on image sensor is characterized by: The method comprises: Set up Eustachian tube stretching training tasks and build a Eustachian tube stretching training VR scene, where each Eustachian tube stretching training task includes a corresponding standard tympanic membrane motion model; The user wears a VR device and selects the Eustachian tube stretching training task in the current Eustachian tube stretching training VR scene, which displays the corresponding matching eardrum standard movement model. collecting tympanic membrane image data corresponding to the user during the Eustachian tube stretching training task using an image sensor; Extracting tympanic membrane dynamic characteristic parameters from the tympanic membrane image data, including tympanic membrane displacement, deformation curvature, and response frequency; The dynamic characteristic parameters of the eardrum are compared with the matching standard motion model of the eardrum, a training score result is output, and the training score result is displayed interactively in VR.

2. The method according to claim 1, wherein According to the dynamic characteristic parameters of the eardrum and the matching eardrum standard motion model, the training score result is output, and the method include: The matching eardrum standard motion model includes a preset reference displacement range, a deformation curvature threshold, and a response frequency interval; Comparing the eardrum displacement, deformation curvature, and response frequency of the eardrum dynamic characteristic parameters with the preset reference displacement range, deformation curvature threshold, and response frequency interval, respectively, and outputting a displacement matching degree, a response timing deviation, and a frequency similarity; The displacement matching degree, response timing deviation and frequency similarity are weightedly calculated, and a training score result is output.

3. The method according to claim 1, wherein The image sensor is built into the ear canal wearing device at one end close to the inner auditory canal opening, and the ear canal wearing device is connected to the VR virtual device via a wireless communication module; The eardrum image data is transmitted to a data processing terminal of the VR virtual device, and the eardrum dynamic feature parameters of the eardrum image data are extracted at the data processing terminal.

4. The method according to claim 3, wherein The ear canal wearing device further comprises a pressure sensor, which is disposed at one end of the ear canal wearing device close to the external auditory canal opening; collecting, according to the pressure sensor, blowing airflow data corresponding to the user during the Eustachian tube blowing training task; The blowing airflow data is transmitted to the data processing terminal of the VR virtual device, the airflow characteristic parameters of the blowing airflow data are extracted at the data processing terminal, and the training score result is outputted by auxiliary calculation according to the airflow characteristic parameters.

5. The method according to claim 4, wherein The method of assisting in calculating and outputting a training score result according to the airflow characteristic parameters includes: Establishing a time sequence mapping relationship between the tympanic membrane dynamic characteristic parameter and the airflow characteristic parameter; A response delay index is calculated according to the timing mapping relationship, and a training score result is updated according to the response delay index.

6. The method according to claim 1, wherein Construct a VR scene for Eustachian tube stretching training, including the following methods: Scanning a healthy Eustachian tube anatomical structure to create three-dimensional point cloud data, wherein the Eustachian tube anatomical structure includes the nasal cavity, nasopharynx, Eustachian tube, middle ear, external auditory canal, and eardrum in the otorhinolaryngology region; Importing the three-dimensional point cloud data into a three-dimensional model, and outputting a three-dimensional model of the Eustachian tube anatomy; Acquire various Eustachian tube stretching training tasks, and obtain various motion parameter libraries for the various Eustachian tube stretching training tasks; Various motion parameter libraries are respectively imported into the Eustachian tube anatomical three-dimensional model to simulate the dynamic changes of the tympanic membrane, and various standard motion models of the tympanic membrane are output, and integrated to obtain a Eustachian tube expansion training VR scene.

7. The method according to claim 6, wherein For each Eustachian tube stretching training task, a library of various exercise parameters is obtained, and the method includes: Eustachian tube stretching movement samples for each Eustachian tube stretching training task; Collecting corresponding eardrum image sample data of multiple user groups under Eustachian tube blowing action samples; Extracting eardrum dynamic feature samples from the eardrum image sample data, including preset reference displacement range samples, deformation curvature threshold samples, and response frequency interval samples; According to the tympanic membrane dynamic feature samples corresponding to each Eustachian tube stretching training task, various motion parameter libraries are generated.

8. The method according to claim 2, wherein Comparing the eardrum displacement, deformation curvature, and response frequency of the eardrum dynamic characteristic parameters with the preset reference displacement range, deformation curvature threshold, and response frequency interval, respectively; wherein, performing Euclidean distance calculation on the eardrum displacement within the preset reference displacement range and outputting the displacement matching degree; Performing DTW dynamic warping algorithm analysis on the deformation curvature within the deformation curvature threshold, and outputting a response timing deviation; Perform Fourier transform on the response frequency and cosine similarity analysis on the response frequency interval to output frequency similarity.

9. The method according to claim 1, wherein The Eustachian tube stretching training task is managed in multiple levels through a permission module, wherein the training difficulty of each level of the Eustachian tube stretching training task increases successively; Recording the user's training score data; If the average of the user's training score data is greater than the preset score average, the training permission for the next level Eustachian tube expansion training task is unlocked.

10. The image sensor-based VR training system for Eustachian tube expansion is characterized by: The system comprises: A training scenario construction module, wherein the training scenario construction module is used to set a Eustachian tube insufflation training task and construct a Eustachian tube insufflation training VR scene, wherein each Eustachian tube insufflation training task includes a corresponding standard tympanic membrane motion model; A training task matching module is used for a user to select a Eustachian tube insufflation training task in the current Eustachian tube insufflation training VR scene by wearing a VR virtual device, and display a corresponding matching tympanic membrane standard motion model; a tympanic membrane image data acquisition module, the tympanic membrane image data acquisition module being configured to acquire tympanic membrane image data corresponding to the user during the Eustachian tube inflating training task using an image sensor; A dynamic feature parameter extraction module, the dynamic feature parameter extraction module is used to extract the eardrum dynamic feature parameters of the eardrum image data, including eardrum displacement, deformation curvature and response frequency; A training score interaction module is used to compare the tympanic membrane dynamic feature parameters with the matching tympanic membrane standard motion model, output a training score result, and perform VR interactive display on the training score result.

Images