Immersive healing system combining virtual reality scene and directional music sequence

Abstract

The invention discloses an immersive healing system combining a virtual reality scene and a directional music sequence, and belongs to the technical field of virtual reality. The system comprises an audio and video synchronization judgment module, a synchronization stability detection module and an audio and video synchronization correction module. Wherein the audio and video synchronization judgment module is beneficial to realizing high-precision synchronization judgment through dynamic monitoring and difference calculation of audio and video frame timestamps. Furthermore, a change rate detection mechanism is introduced into the synchronous stability detection module, so that synchronous unbalance caused by slow accumulative errors in the healing process is effectively avoided; and meanwhile, the audio and video synchronous correction module is used for performing audio delay correction and visual delay correction according to the synchronous time difference, so that smooth synchronous repair is realized on the premise of not destroying the experience continuity of the healing path. Through closed-loop cooperation of the three modules, accurate synchronization of a virtual reality scene and a directional music sequence in a healing path is realized, and immersion in the healing process is further significantly improved.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, and in particular to an immersive healing system that combines virtual reality scenes with directional music sequences. Background Technology

[0002] In today's society, with the accelerating pace of life and increasing social pressure, mental health issues are becoming increasingly prominent, and people's demand for emotion regulation and psychological healing has risen significantly. On the one hand, music therapy, as a widely validated non-pharmacological psychological intervention, can regulate a person's emotions and physiological responses through sound stimulation of specific frequencies and rhythms. On the other hand, the rapid development of virtual reality (VR) technology has provided new possibilities for psychological healing. When directional music is combined with immersive virtual reality scenarios, a synergistic effect can be created at the visual, auditory, and psychological levels, strengthening emotional resonance and enhancing the immersion and effectiveness of the healing experience.

[0003] Existing immersive therapy technologies combining virtual reality scenarios and directional music sequences primarily utilize multimodal perception and intelligent matching algorithms to identify, intervene in, and regulate users' emotional states. The overall process typically includes user information collection, emotion analysis and modeling, scenario and music generation, immersive interactive experience, and data feedback optimization. Specifically, it first acquires users' psychological and physiological data through psychological testing, physiological signals, or emotion recognition technology, and establishes an individualized emotion model. Then, based on the analysis results, it automatically generates or matches virtual reality therapy scenarios and directional music sequences that resonate with the user's current emotions. Users experience both visual and auditory sensations in an immersive environment, while simultaneously adjusting lighting effects, colors, and music rhythms through real-time monitoring of physiological indicators to maintain optimal relaxation. After the therapy session, a therapy report is automatically generated and personalized optimizations are performed to achieve continuous emotional improvement and psychological regulation. Overall, this technology, through the deep integration of virtual reality and music therapy, constructs a perceptible, interactive, and adaptive digital therapy system, providing a new intelligent and immersive approach to modern mental health intervention.

[0004] For example, the virtual reality scene fusion and playback method and virtual reality device disclosed in Chinese invention patent CN113256815B include: acquiring dynamic three-dimensional mesh data of at least one actual scene, and acquiring stereo sound field data of at least one actual scene; fusing the dynamic three-dimensional mesh data and stereo sound field data into the user's virtual space to obtain a virtual fused scene; acquiring the user's state information in the virtual fused scene; obtaining the target visual information and target acoustic information corresponding to the virtual fused scene based on the state information; and playing the target visual information and target acoustic information.

[0005] However, in the process of implementing the inventive technical solution in the embodiments of this application, it was found that the above-mentioned technology has at least the following technical problems:

[0006] In immersive therapy combining virtual reality scenarios with directional music sequences, the interaction between the music sequence and the virtual environment needs to be highly synchronized to ensure effective emotional and psychological regulation. However, frame rate fluctuations in virtual reality can cause the screen to update untimely when the user moves or rotates their viewpoint rapidly, resulting in "jittering" or "stuttering" in the virtual environment. Especially during therapy sessions, when users expect smooth and stable visual feedback to help them relax, frame rate instability can increase user tension and even cause visual fatigue. Prolonged exposure to this unstable state can lead to discomfort such as dizziness and eye strain, and in severe cases, vertigo or nausea, disrupting the continuity and smoothness of the virtual environment and further weakening the immersive therapy effect. Summary of the Invention

[0007] To address the technical problem of low synchronization and smoothness in virtual reality scenes, leading to poor immersive healing effects in existing technologies, this invention provides an immersive healing system that combines virtual reality scenes with directional music sequences. The technical solution is as follows:

[0008] An immersive healing system combining virtual reality scenes and directional music sequences includes: an audio-visual synchronization judgment module, used to acquire rendering frame timestamps and audio frame timestamps in the virtual reality scene under a preset healing path, calculate a synchronization time difference based on the rendering frame timestamps and audio frame timestamps, and determine whether the synchronization time difference is within a set synchronization threshold range. If it is, the operation of the synchronization stability detection module is triggered; otherwise, the operation of the audio-visual synchronization correction module is triggered. A synchronization stability detection module is used to quantify the changing trend of the synchronization time difference to obtain the synchronization time difference change rate, and determine whether the synchronization time difference change rate triggers the synchronization trend execution decision condition. If it is triggered, the operation of the audio-visual synchronization correction module is executed; otherwise, the operation of the audio-visual synchronization judgment module is re-executed. An audio-visual synchronization correction module is used to first perform audio delay correction based on the synchronization time difference to adjust the audio playback rate and audio buffer duration of the directional music sequence under the healing path, and then re-acquire the synchronization time difference of the next sliding window. If the re-acquired synchronization time difference is within a set synchronization threshold range, the system returns and triggers the operation of the synchronization stability detection module; otherwise, visual delay correction is performed to adjust the rendering frame rate of the virtual reality scene under the healing path.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0010] This invention achieves dynamic temporal synchronization control of virtual reality scene rendering and directional music playback through the collaborative operation of multiple modules, thereby ensuring audio-visual consistency and experience stability during immersive healing processes. Specifically, the audio-visual synchronization judgment module can monitor audio-visual synchronization deviations in real time, ensuring accurate time correspondence between virtual reality visuals and music playback, providing precise basis for subsequent synchronization trend detection and correction, thus improving rhythmic coordination and immersion during the healing process. Furthermore, the synchronization stability detection module can dynamically track the changing trend of audio-visual synchronization status, achieving early identification and trend prediction of synchronization deviations, thereby proactively correcting before asynchrony significantly affects the experience, effectively maintaining audio-visual synchronization stability during the healing process. Simultaneously, the audio-visual synchronization correction module adopts a layered correction strategy, prioritizing lightweight audio adjustments, and then combining them with visual rendering corrections, thereby achieving smooth synchronization repair without disrupting the continuity of the healing path experience, enhancing the adaptability and immersion of complex dynamic scenes. Compared with existing technologies, this invention can effectively solve the problems of poor audio-visual synchronization and low playback smoothness in virtual reality scenes, fundamentally improve the sensory misalignment and experience fragmentation that occur during immersive healing, and enhance the immersive effect of healing content.

[0011] In summary, this invention organically combines an audio-visual synchronization judgment module, a synchronization stability detection module, and an audio-visual synchronization correction module to form a closed-loop dynamic audio-visual synchronization control system. This system enables high-precision audio-visual synchronization and adaptive adjustment in immersive therapeutic scenarios, not only improving the integration and smoothness of virtual reality and music therapy, but also significantly reducing interruptions to immersion and psychological discomfort caused by synchronization discrepancies, thereby enhancing the user's emotional relaxation and the quality of the therapeutic experience. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram of the structure of an immersive healing system combining a virtual reality scene and a directional music sequence, provided in an embodiment of this application;

[0014] Figure 2 A threshold range correction setting logic diagram for an immersive healing system combining virtual reality scenes and directional music sequences, provided in an embodiment of this application;

[0015] Figure 3This is a flowchart illustrating the audio delay correction process of an immersive healing system that combines a virtual reality scene with a directional music sequence, as provided in an embodiment of this application. Detailed Implementation

[0016] Embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present disclosure are shown in the drawings, it should be understood that embodiments of the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure.

[0017] It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure. In the description of the embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "this embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] With increasing pressure in modern society, mental health issues (such as anxiety, depression, and stress) have gradually become a significant public health problem worldwide. According to statistics from the World Health Organization, the number of people with mental health problems is increasing year by year globally, especially in highly urbanized areas where the pace of life is fast and social pressure is high, resulting in a persistently high incidence of mental health issues.

[0020] Traditional psychotherapies, such as cognitive behavioral therapy or drug therapy, sometimes fail to meet the full needs of patients, especially in terms of emotional support and relaxation. Therefore, an innovative healing approach is needed to supplement existing treatments. Designing an immersive healing system that combines virtual reality scenarios with directional music sequences is not only a practical necessity for addressing contemporary mental health issues but also a significant innovative direction for technology-enabled psychotherapy. It not only helps improve the scientific rigor and experiential aspects of psychological intervention but also provides a feasible technological path and social value for the future development of digital mental health services.

[0021] like Figure 1 The diagram shown is a structural schematic of an immersive healing system combining a virtual reality scene and a directional music sequence, provided in an embodiment of this application. (Refer to...) Figure 1The system includes an audio-visual synchronization judgment module, a synchronization stability detection module, and an audio-visual synchronization correction module. Through the collaborative work of these modules, adaptive dynamic synchronization and precise and stable control of audio and video in immersive therapeutic scenarios are achieved.

[0022] The audio-visual synchronization judgment module provides a basic synchronization monitoring mechanism by calculating the time difference between the virtual reality scene rendering frame and the directional music sequence audio frame in real time. The synchronization stability detection module further quantifies the trend of time difference changes to achieve dynamic evaluation and prediction of synchronization status, thereby avoiding the accumulation of errors caused by judging solely by instantaneous time difference. The audio-visual synchronization correction module performs precise correction through a two-way adjustment mechanism after detecting deviation, that is, first fine-tuning the audio playback rate and buffering time, and then compensating for the visual frame rate when necessary.

[0023] The coordinated operation of each module not only forms a closed-loop feedback dynamic synchronous control system, but also significantly improves the coordination and continuity of audio and video output, effectively avoiding phenomena such as image and sound misalignment and delay drift. Through this mechanism, the present invention can sustainably maintain a high-precision match between the virtual reality healing scene and the directional music sequence, thereby enhancing immersion, stabilizing the healing rhythm, and significantly improving the user's psychological relaxation and immersive experience.

[0024] Specifically, the first module of an immersive healing system combining virtual reality scenes and directional music sequences is the audio-visual synchronization judgment module. This module obtains the rendering frame timestamps and audio frame timestamps within the virtual reality scene under a preset healing path. Specifically, it obtains the rendering frame timestamps through a frame rendering time callback function provided by a graphics rendering interface (such as OpenGL) and the audio frame timestamps through an audio engine interface (such as FMOD). Next, it calculates the synchronization time difference based on the rendering and audio frame timestamps. Then, it determines whether the synchronization time difference is within a set synchronization threshold range. If it is, it triggers the operation of the synchronization stability detection module; otherwise, it triggers the operation of the audio-visual synchronization correction module. This module effectively improves the system's real-time perception capability of audio-visual synchronization status, providing reliable basic data for subsequent stability detection and correction.

[0025] It should be noted that in the audio-video synchronization judgment module, the synchronization time difference is calculated based on the rendering frame timestamp and the audio frame timestamp. The specific steps are as follows:

[0026] First, the average synchronization time difference is obtained by averaging the absolute differences between the rendering frame timestamp and the audio frame timestamp corresponding to each frame number within the sliding window. Then, the obtained average synchronization time difference is coupled and corrected with the introduced frame synchronization drift correction factor to obtain the synchronization time difference. The coupling correction process is a product operation. Specifically, the frame synchronization drift correction factor is used to compensate for the cumulative error of synchronization deviation caused by the asynchrony of device clocks. It is set by professionals according to the standards in the field.

[0027] It is important to understand that, because audio playback and video rendering in virtual reality scenarios are often driven by different clocks, there is a slight frequency deviation between the two, which accumulates over time and generates synchronization drift errors. To address this, this invention introduces a frame synchronization drift correction factor into the audio-video synchronization judgment module to couple and correct the average synchronization time difference within the sliding window. By introducing this correction factor, the accumulated synchronization deviation error caused by asynchronous device clocks can be effectively compensated, making the synchronization time difference calculation more stable and accurate. This maintains high-precision audio-video synchronization during long playback periods, ensuring the rhythmic consistency of visual images and directional music in virtual reality healing scenarios, and significantly improving the continuity and comfort of the immersive healing experience.

[0028] For example, the frame synchronization drift correction factor is dynamically adjusted based on the clock difference between the audio playback device and the video rendering device to adapt to different degrees of device clock offset and ensure the accuracy and stability of audio-video synchronization calculation. Specifically: when the clock difference between devices is less than 1 millisecond, the current audio-video synchronization state is considered relatively stable. In this case, to avoid unnecessary disturbance to the synchronization system, the frame synchronization drift correction factor is set to 0.05. When the clock difference between devices is between 1 and 5 milliseconds, it indicates that there is a certain clock frequency deviation between the devices, which may cause slight synchronization errors during long-term playback. In this case, to achieve appropriate time difference compensation, the frame synchronization drift correction factor is set to 0.1 to 0.2 to achieve a balance between correction effect and system stability. When the clock difference between devices is in the range of 5 to 10 milliseconds, it indicates that the clock reference difference between the audio and video devices is relatively significant, and the audio-video synchronization deviation may gradually accumulate. In this case, to effectively suppress the expansion of synchronization error, the frame synchronization drift correction factor is set to 0.3 to 0.4 to enhance the response strength of synchronization correction. When the clock difference between devices exceeds 10 milliseconds, it is determined that the clock asynchrony between the devices is relatively high. If not corrected in time, it may lead to obvious audio and video desynchronization. At this time, the frame synchronization drift correction factor is set to 0.5 to quickly compensate for the synchronization time difference caused by clock offset and realize the timely restoration of audio and video synchronization.

[0029] By introducing a sliding window-based synchronization time difference calculation mechanism and a frame synchronization drift correction factor into the audio-visual synchronization judgment module, high-precision time synchronization control between virtual reality scene rendering frames and directional music audio frames is achieved. Furthermore, by introducing a frame synchronization drift correction factor, errors caused by clock offsets can be estimated and dynamically corrected in real time, thus maintaining precise alignment between the rendering frame timestamp and the audio frame timestamp during long-term system operation. This correction factor is coupled with the average synchronization time difference in the synchronization time difference calculation, enabling fine-grained compensation for audio-visual synchronization drift. This allows the system to maintain high-stability audio-visual synchronization even in multi-device collaborative operation or cross-platform playback environments. This correction mechanism helps maintain long-term coordination between visual changes and music rhythm in virtual reality healing scenarios, avoiding audio-visual misalignment caused by accumulated delays, and significantly improving user immersion and comfort.

[0030] As a further solution, due to the dynamic factors affecting the operation of virtual reality healing systems, such as rendering computational load, audio playback caching, device clock deviation, and network latency, the audio-video synchronization state exhibits periodic fluctuations. Using a fixed threshold for judgment makes it difficult to simultaneously ensure system stability and response speed, easily leading to over-correction or lag. Therefore, by introducing an adaptive threshold correction mechanism based on synchronization variance statistics, the synchronization judgment threshold can be kept stable when the system is in a stable state, while the threshold range can be adjusted promptly when significant system jitter occurs, preventing the continuous accumulation of audio-video desynchronization. Therefore, the audio-video synchronization judgment module determines whether the synchronization time difference is within the set synchronization threshold range, and then further determines whether to adjust the synchronization threshold range setting, specifically:

[0031] First, the synchronization time difference of each sliding window within the healing time period is statistically analyzed to obtain the synchronization variance. Second, it is determined whether the obtained synchronization variance is within the synchronization jitter interval. If so, no correction setting of the synchronization threshold interval is performed. The synchronization jitter interval is obtained from the healing management database and is a closed interval corresponding to the minimum variance threshold to the maximum variance threshold. The healing management database is a database created specifically to store core configuration information when designing an immersive healing system that combines virtual reality scenes and directional music sequences. This database stores various thresholds necessary for the operation of the system, such as the minimum variance threshold and the maximum variance threshold. The initial settings of these thresholds are not arbitrarily specified. Technicians can manually set, adjust, or fine-tune them at any time according to the specific performance of the system in actual testing.

[0032] If the synchronization variance is not within the synchronization jitter interval, the synchronization threshold interval is adjusted based on the numerical relationship between the synchronization variance and the synchronization jitter interval. Figure 2The diagram shown is a threshold range correction setting logic diagram for an immersive healing system combining virtual reality scenes and directional music sequences provided in an embodiment of this application. (Refer to...) Figure 2 The specific process is as follows:

[0033] The synchronization variance is compared numerically with the minimum variance threshold and the maximum variance threshold, and the following decision is made based on the comparison results:

[0034] (1) When the synchronization variance is less than the minimum variance threshold, the shrinkage threshold adjustment factor corresponding to the synchronization variance is queried according to the pre-stored mapping relationship. The set synchronization threshold interval is updated after shrinkage coupling correction based on the shrinkage threshold adjustment factor. The shrinkage coupling correction here is to multiply the minimum variance threshold and the maximum variance threshold with the shrinkage threshold adjustment factor respectively.

[0035] (2) When the synchronization variance is greater than the maximum variance threshold, the extended threshold adjustment factor corresponding to the synchronization variance is obtained from the pre-stored mapping relationship. The extended coupling correction is performed on the set synchronization threshold interval based on the extended threshold adjustment factor and then updated. The extended coupling correction here is to multiply the minimum variance threshold and the maximum variance threshold with the extended threshold adjustment factor respectively.

[0036] Through the dynamic switching of the above two correction mechanisms of contraction and expansion, the system can maintain the rationality of the synchronization threshold range under different usage states, so that the visual changes of the virtual reality scene and the acoustic rhythm of the directional music are coordinated, thereby taking into account stability and immersion during the healing process, improving the overall experience, and further ensuring the continuity of audio-visual output and immersive experience in the virtual reality healing scene.

[0037] The second module of this immersive healing system, which combines virtual reality scenes with directional music sequences, is a synchronization stability detection module. This module quantifies the changing trend of the synchronization time difference to obtain the synchronization time difference change rate. It then determines whether the change rate triggers the synchronization trend execution decision condition. If triggered, the audio-visual synchronization correction module is executed; otherwise, the audio-visual synchronization judgment module is re-executed. This module effectively identifies synchronization drift trends, enabling early prediction and preventative control. This avoids system fluctuations caused by frequent corrections, improving the smoothness and stability of system operation.

[0038] It should be noted that the specific process for determining whether the rate of change of synchronization time difference in the synchronization stability detection module triggers the synchronization trend execution decision condition is as follows:

[0039] First, obtain the rate of change of synchronization time difference, which is the result of differentiating the synchronization time difference within the sliding window. Second, if the rate of change of synchronization time difference is within the set synchronization trend limit range, the synchronization trend execution decision condition is not triggered; otherwise, the synchronization trend execution decision condition is triggered and an abnormal synchronization trend prompt is sent. The synchronization trend limit range is obtained from the healing management database.

[0040] As a further solution, since the audio-visual synchronization status may vary under different healing stages, network transmission conditions, and device operating loads, if the sliding window step size and the total number of sampling frames remain fixed, problems may arise such as excessively high sampling frequency leading to system computational redundancy and decreased real-time performance, or excessively low sampling frequency leading to synchronization detection lag and insufficient accuracy. Therefore, this invention introduces an adaptive adjustment mechanism for the sliding window and the number of sampling frames based on the rate of change of synchronization time difference in the synchronization stability detection module, enabling the system to automatically adjust the sampling granularity and analysis interval according to the speed of synchronization changes. Specifically, the synchronization stability detection module further includes the following step before re-executing the audio-visual synchronization judgment module:

[0041] Based on the pre-stored mapping relationship, query the sliding window step size adjustment amount and the total number of sampling adjustment frames corresponding to the synchronization time difference change rate; adjust the sliding window and the total number of sampling frames according to the obtained sliding window step size adjustment amount and the total number of sampling adjustment frames, and then re-execute the operation of the audio and video synchronization judgment module.

[0042] By dynamically adjusting the sliding window and the total number of sampling frames, the sliding window step size can be automatically shortened and the number of sampling frames increased when the synchronization rate of change is large, thereby improving the temporal resolution and response speed of synchronous detection. When the synchronization rate of change is small, the sliding window step size is automatically increased and the number of sampling frames is reduced, thereby reducing the computational load and improving detection stability. This mechanism can ensure detection accuracy while taking into account the real-time response performance of the system, effectively avoiding over-computation or detection delay caused by fixed parameter sampling.

[0043] Through the above design, the present invention achieves adaptive optimization of the sampling strategy in the synchronization stability detection stage, making the audio-visual synchronization judgment process more flexible, efficient and robust, further improving the system's self-adjustment capability and overall robustness under different operating conditions, and ensuring that the audio-visual synchronization in immersive healing scenarios always remains smooth, natural and coordinated.

[0044] The third module of this immersive healing system, which combines virtual reality scenes with directional music sequences, is the audio-visual synchronization correction module. This module first corrects audio latency based on the synchronization time difference to adjust the audio playback rate and audio buffer duration of the directional music sequence within the healing path. Then, it re-acquires the synchronization time difference for the next sliding window. If the re-acquired synchronization time difference is within a set synchronization threshold range, it returns and triggers the synchronization stability detection module. Otherwise, it performs visual latency correction to adjust the rendering frame rate of the virtual reality scene within the healing path. By introducing a graded correction mechanism, this module achieves multi-dimensional synchronization correction, effectively eliminating audio-visual asynchrony and ensuring rhythmic coordination and sensory consistency throughout the healing process.

[0045] It should also be noted that, such as Figure 3 The diagram shown is an audio delay correction flowchart for an immersive healing system combining a virtual reality scene and a directional music sequence, provided in an embodiment of this application. (Refer to...) Figure 3 In the audio / video synchronization correction module, the specific steps for audio delay correction are as follows:

[0046] First, determine the relationship between the audio frame timestamp and the rendering frame timestamp. If the audio frame timestamp is smaller than the rendering frame timestamp, input the synchronization time difference to the pre-stored mapping relationship to output the audio playback rate reduction amount and the buffer duration increase amount respectively. Adjust the audio playback rate and audio buffer duration during the playback of the directional music sequence according to the output audio playback rate reduction amount and buffer duration increase amount respectively. That is, decrease the audio playback rate with the step size corresponding to the audio playback rate reduction amount and increase the audio buffer duration with the step size corresponding to the buffer duration increase amount.

[0047] Conversely, based on the pre-stored mapping relationship, the audio playback rate increase and buffer duration decrease corresponding to the synchronization time difference are obtained, and the audio playback rate and audio buffer duration are adjusted accordingly based on the obtained audio playback rate increase and buffer duration decrease; that is, the audio playback rate is increased by the step size corresponding to the audio playback rate increase, and the audio buffer duration is decreased by the step size corresponding to the buffer duration decrease.

[0048] It's important to understand that by setting up an audio delay correction mechanism within the audio-video synchronization correction module, the audio playback rate and audio buffer duration can be dynamically adjusted based on the relationship between the audio frame timestamp and the rendering frame timestamp, utilizing a pre-stored mapping relationship to achieve precise matching between audio playback progress and visual rendering progress. Specifically, when audio playback is ahead, reducing the playback rate and extending the buffer duration smoothly delays the audio output, avoiding abrupt pauses or jumps; when audio playback is behind, increasing the playback rate and shortening the buffer duration quickly catches up with the visual rhythm, thus restoring synchronization in a short time. This correction method makes the audio synchronization adjustment process directionally adaptive and amplitude controllable, not only improving the real-time performance and stability of synchronization correction but also effectively avoiding the sound quality distortion and playback discontinuity problems caused by traditional single-parameter rate correction. Through this mechanism, the system can maintain the consistency and smooth transition of audio-visual rhythm under different delay conditions, significantly enhancing the sound continuity, spatial sense, and comfort in immersive healing scenarios.

[0049] Furthermore, since both playback rate and buffer duration are dynamic parameters during audio delay correction, their adjustment range is limited by various factors such as system performance, buffer capacity, and audio decoding stability. Excessive adjustment may cause audio signal distortion, interruptions, or new synchronization deviations with the video output; conversely, insufficient adjustment may result in slow synchronization recovery or ineffective correction. Therefore, it is necessary to monitor the adjustment results and provide status indicators to ensure the correction process remains within a controllable range. This means that after audio delay correction, the following steps are also required:

[0050] First, obtain the adjustment range of the adjusted audio playback rate and audio buffer duration, and determine whether the obtained adjustment ranges are within the corresponding adjustment range limits. If so, send an audio delay correction normal prompt; otherwise, send an audio delay correction abnormal prompt.

[0051] By setting up this detection and prompting mechanism, the system can automatically verify the rationality of parameter adjustments after completing audio delay correction, and realize self-checking and feedback of the correction results.

[0052] It should also be noted that the specific steps for visual delay correction in the audio / video synchronization correction module are as follows:

[0053] If the rendering frame timestamp of the next sliding window is greater than the audio frame timestamp, the rendering frame rate reduction amount corresponding to the synchronization time difference is obtained from the pre-stored mapping relationship, and the corresponding rendering frame rate in the virtual reality scene is adjusted accordingly, that is, the rendering frame rate is reduced by the step size corresponding to the rendering frame rate reduction amount; conversely, the rendering frame rate increase amount corresponding to the synchronization time difference is obtained from the pre-stored mapping relationship, and the rendering frame rate of the virtual reality scene is adjusted according to the obtained rendering frame rate increase amount, that is, the rendering frame rate is increased by the step size corresponding to the rendering frame rate increase amount.

[0054] By introducing a visual delay correction mechanism into the audio-video synchronization correction module, when the rendering frame timestamp of the next sliding window is detected to be greater than the audio frame timestamp, the system can obtain the rendering frame rate reduction amount corresponding to the synchronization time difference according to the pre-stored mapping relationship, and appropriately reduce the rendering frame rate of the virtual reality scene, thereby correcting the video output delay; when the rendering frame timestamp is less than the audio frame timestamp, the corresponding rendering frame rate increase amount is obtained according to the synchronization time difference, and the rendering frame rate of the virtual reality scene is increased, so that the video output and audio signal are synchronized again.

[0055] Through the aforementioned dynamic adjustment process, visual lag caused by system latency differences during audio and video playback can be effectively eliminated, achieving real-time matching and coordination of audio and video content, and significantly improving the consistency and naturalness of the immersive experience. Simultaneously, frame rate adjustment is based on pre-stored mapping relationships, ensuring smooth visuals while avoiding performance overhead caused by frequent calculations, thus balancing the system's real-time performance and stability. This mechanism can significantly improve user immersion and comfort in virtual reality therapy scenarios, further enhancing the overall effect of the therapy system.

[0056] Furthermore, visual delay correction is performed, which also includes:

[0057] Determine whether the adjusted rendering frame rate is within the set rendering frame rate limit range. If not, send a visual delay correction error message; if so, send a visual delay correction normal message and re-execute the audio and video synchronization judgment module.

[0058] After completing the visual delay correction, the system further detects the adjusted rendering frame rate to determine if it is within the preset rendering frame rate limit range. If the detection result shows that the rendering frame rate exceeds the limit range, the system sends a visual delay correction anomaly prompt to indicate that there is a deviation in the current synchronization adjustment or that the hardware performance is limited, thus providing a basis for subsequent system stability optimization; if the rendering frame rate is within the set limit range, the system sends a visual delay correction normal prompt and re-executes the operation of the audio and video synchronization judgment module to achieve continuous audio and video synchronization monitoring and dynamic correction.

[0059] This detection and feedback mechanism further ensures the stability and controllability of the system operation on top of visual latency correction, preventing issues such as screen jitter, stuttering, or abnormal system resource usage caused by excessive frame rate adjustment. Through closed-loop detection and feedback, the mechanism achieves self-correction and operational status monitoring of the audio-visual synchronization process, ensuring the entire immersive healing system maintains good synchronization accuracy and visual smoothness during long-term operation, thereby further enhancing the user's immersive experience and therapeutic effect in virtual reality scenarios.

[0060] This invention achieves precise time synchronization control between virtual reality rendered visuals and directional music playback by setting up an audio-visual synchronization judgment module, a synchronization stability detection module, and an audio-visual synchronization correction module. The system can detect the difference between the rendering frame timestamp and the audio frame timestamp in real time within the dynamic scene of the healing path, and adaptively correct based on the synchronization time difference, thereby effectively avoiding the immersion disruption caused by audio-visual asynchrony. By introducing a synchronization stability detection mechanism, this system can not only quickly correct synchronization deviations when they occur, but also quantitatively judge synchronization change trends, predicting and intervening in potential out-of-synchrony risks in advance. The audio-visual synchronization correction module adopts a hierarchical correction strategy, prioritizing fine-tuning of audio latency, and then dynamically adjusting the visual rendering frame rate when necessary, thus helping to maintain high-precision audio-visual matching. In summary, this invention significantly improves the immersive experience and sensory coordination in virtual reality healing scenarios, enhances the stability of the healing effect, and improves user psychological comfort.

[0061] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the above functions can be divided into different functional modules to complete all or part of the functions described above.

[0062] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0063] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units, located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0064] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0065] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the solution, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An immersive healing system combining virtual reality scenes and directional music sequences, the system comprising: The audio and video synchronization judgment module is used to obtain the rendering frame timestamp and audio frame timestamp in the virtual reality scene under the preset healing path, calculate the synchronization time difference based on the rendering frame timestamp and audio frame timestamp, and determine whether the synchronization time difference is within the set synchronization threshold range. If it is, the operation of the synchronization stability detection module is triggered; otherwise, the operation of the audio and video synchronization correction module is triggered. The synchronization stability detection module is used to quantify the changing trend of the synchronization time difference to obtain the synchronization time difference change rate, and determine whether the synchronization time difference change rate triggers the synchronization trend execution decision condition. If it is triggered, the operation of the audio and video synchronization correction module is executed; otherwise, the operation of the audio and video synchronization judgment module is re-executed. The audio and video synchronization correction module is used to first correct the audio delay based on the synchronization time difference to adjust the audio playback rate and audio buffer duration of the directional music sequence under the healing path, and then re-acquire the synchronization time difference of the next sliding window. If the re-acquired synchronization time difference is within the set synchronization threshold range, it returns and triggers the operation of the synchronization stability detection module. Otherwise, it performs visual delay correction to adjust the rendering frame rate of the virtual reality scene under the healing path.

2. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, In the audio-video synchronization judgment module, the synchronization time difference is calculated based on the rendering frame timestamp and the audio frame timestamp. The specific steps are as follows: The average synchronization time difference is obtained by averaging the absolute differences between the rendering frame timestamp and the audio frame timestamp corresponding to each frame number within the sliding window. The obtained average synchronization time difference is coupled with the introduced frame synchronization drift correction factor to obtain the synchronization time difference; The frame synchronization drift correction factor is used to compensate for the cumulative error caused by synchronization deviation due to asynchronous device clocks.

3. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, The audio / video synchronization judgment module determines whether the synchronization time difference is within a set synchronization threshold range, and then further determines whether to adjust the synchronization threshold range, specifically: The synchronization variance was obtained by statistically analyzing the synchronization time difference of each sliding window within the healing time period. Determine whether the obtained synchronization variance is within the synchronization jitter interval. If so, no correction setting is made for the synchronization threshold interval. The synchronization jitter interval is the closed interval corresponding to the minimum variance threshold to the maximum variance threshold. Conversely, the synchronization threshold range is adjusted and set based on the numerical relationship between the synchronization variance and the synchronization jitter range.

4. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 3, characterized in that, The specific process for setting and correcting the synchronization threshold range is as follows: The synchronization variance is compared numerically with the minimum variance threshold and the maximum variance threshold, and the following decision is made based on the comparison results: (1) When the synchronization variance is less than the minimum variance threshold, the shrinkage threshold adjustment factor corresponding to the synchronization variance is queried according to the pre-stored mapping relationship, and the set synchronization threshold interval is updated after shrinkage coupling correction based on the shrinkage threshold adjustment factor. (2) When the synchronization variance is greater than the maximum variance threshold, the extended threshold adjustment factor corresponding to the synchronization variance is obtained, and the set synchronization threshold interval is updated after extended coupling correction based on the extended threshold adjustment factor.

5. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, In the synchronization stability detection module, the process for determining whether the rate of change of synchronization time difference triggers the synchronization trend execution decision condition is as follows: The rate of change of synchronization time difference is obtained, which is the result of taking the derivative of the synchronization time difference within the sliding window. If the rate of change of the synchronization time difference is within the set synchronization trend limit range, the synchronization trend execution decision condition will not be triggered; otherwise, the synchronization trend execution decision condition will be triggered and an abnormal synchronization trend prompt will be sent.

6. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, In the synchronization stability detection module, the operation of the audio-video synchronization judgment module is re-executed, which previously included: Based on the pre-stored mapping relationship, query the sliding window step size adjustment amount and the total number of sampling adjustment frames corresponding to the rate of change of synchronization time difference; After adjusting the sliding window step size and the total number of sampling frames based on the obtained sliding window step size and the total number of sampling frames, the operation of the audio and video synchronization judgment module is re-executed.

7. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, The specific steps for audio delay correction in the audio / video synchronization correction module are as follows: Determine the relationship between the audio frame timestamp and the rendering frame timestamp. If the audio frame timestamp is smaller than the rendering frame timestamp, input the synchronization time difference to the pre-stored mapping relationship to output the audio playback rate reduction amount and the buffer duration increase amount respectively. The audio playback rate and audio buffer duration during the playback of the directional music sequence are adjusted according to the amount of decrease in the output audio playback rate and the amount of increase in the buffer duration, respectively. Conversely, based on the pre-stored mapping relationship, the corresponding audio playback rate increase and buffer duration decrease are obtained, and the obtained audio playback rate increase and buffer duration decrease are used to adjust the audio playback rate and audio buffer duration accordingly.

8. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 7, characterized in that, The audio delay correction process also includes: The adjustment ranges of the adjusted audio playback rate and audio buffer duration are obtained respectively, and it is determined whether the obtained adjustment ranges are within the corresponding adjustment range limit ranges. If yes, a message indicating that the audio delay correction is normal will be sent; otherwise, a message indicating that the audio delay correction is abnormal will be sent.

9. The immersive healing system combining virtual reality scenes and directional music sequences as described in claim 1, characterized in that, The specific steps for visual delay correction in the audio-visual synchronization correction module are as follows: If the rendering frame timestamp of the next sliding window is greater than the audio frame timestamp, the rendering frame rate reduction amount corresponding to the synchronization time difference is obtained from the pre-stored mapping relationship, and the corresponding rendering frame rate in the virtual reality scene is adjusted. Conversely, the rendering frame rate increase amount corresponding to the synchronization time difference is obtained from the pre-stored mapping relationship, and the rendering frame rate of the virtual reality scene is adjusted according to the obtained rendering frame rate increase amount.

10. An immersive healing system combining virtual reality scenes and directional music sequences as described in claim 9, characterized in that, The process of visual delay correction is followed by: Determine whether the adjusted rendering frame rate is within the set rendering frame rate limit range. If not, send a visual delay correction error message. If so, a visual delay correction notification will be sent, and the audio / video synchronization judgment module will be re-executed.

Images