Immersive game audio adaptive tuning method and system based on deep learning

Abstract

The present application relates to the technical field of immersive audio processing and intelligent audio tuning, in particular to an immersive game audio adaptive tuning method and system based on deep learning, comprising: superimposing a low-energy detection signal in an audio rendering output link, using a near-ear microphone, an in-cabin reference microphone and an inertial measurement unit to obtain left and right ear sound pressure, ambient sound pressure and head posture data; estimating left and right sound path transfer functions, extracting low-frequency leakage, high-frequency positioning deviation, imbalance, early reflection amount and microphone confidence; inputting physical characteristics, environmental noise, program loudness and last period execution parameters into a time series deep learning model, combining error change trend and head angular velocity constraint to obtain final execution parameters, and applying to equalization, head-related transfer function fusion, binaural time difference, binaural level difference and reflection compensation unit to realize immersive game audio adaptive tuning.

Description

Technical Field

[0001] This invention relates to the field of immersive audio processing and intelligent audio tuning technology, specifically to an immersive game audio adaptive tuning method and system based on deep learning. Background Technology

[0002] In immersive game audio rendering scenarios, the headset, headphones, and near-ear pickup components will jointly affect the reproduction effect of the binaural sound field. Furthermore, changes in user head rotation, loosening of headphones, and changes in ambient noise and reflected sound within the cabin will cause continuous fluctuations in the sound pressure distribution, sound path transmission characteristics, and spatial positioning effect of the left and right ears. Therefore, the calibration accuracy of the audio rendering link depends on the ability to perceive and stably control the wearing status, acoustic environment, and binaural physical characteristics in real time. In traditional methods, immersive audio systems often use factory-fixed parameters, initial calibration parameters, or rule-based compensation methods based on a small number of instantaneous features to complete equalization, positioning, and reflection processing. When the user's posture changes continuously, the ambient noise fluctuates, or the headphone fit changes, it is difficult to track the changes in the sound path in a timely and accurate manner and to correct the rendering parameters synchronously. This leads to technical defects such as increased low-frequency leakage, left and right ear imbalance, increased positioning deviation, and parameter oscillation, resulting in low adaptive tuning stability of immersive game audio. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based method and system for adaptive audio tuning in immersive games, addressing the following technical problem: Existing immersive audio systems fail to achieve sound path tracking and synchronous parameter correction when faced with changes in user posture, environmental noise fluctuations, and changes in headphone fit. This invention proposes a method and system capable of real-time perception of multi-dimensional physical characteristics, precise adaptive tuning, and stable control of the rendering link. The objective of this invention can be achieved through the following technical solutions: The deep learning-based adaptive audio tuning method for immersive games superimposes low-energy detection signals into the immersive game audio rendering output link. It uses near-ear microphones, in-cabin reference microphones, and inertial measurement units to acquire sound pressure levels in the left and right ears, ambient sound pressure levels, and head posture data. Based on the ambient sound pressure data, it extracts the ambient noise sound pressure level and frequency band noise energy, and based on the current program audio, it extracts the loudness value of the current program. In the first detection cycle, it uses the initialization parameter set as the execution parameters of the previous cycle, and in subsequent detection cycles, it uses the execution parameters of the previous detection cycle as the execution parameters of the previous cycle. Based on the detection signal and the sound pressure data of the left and right ears, the transfer function of the left and right acoustic paths is estimated. The difference between the amplitude-frequency response of the transfer function of the current cycle and the previous cycle is differentially analyzed to obtain the instantaneous change direction of each frequency band rising or falling. The low-frequency leakage, high-frequency positioning deviation, left and right ear imbalance, early reflection, and microphone confidence are extracted. The physical features, head posture data, environmental noise, current program loudness value and previous cycle execution parameters are combined into an input data sequence, which is then input into a time-series deep learning model that is trained offline and then inferred online. The model outputs initial calibration parameters and retains only the initial calibration parameters that are consistent with the compensation direction corresponding to the instantaneous change direction as candidate parameters. The error change trend is determined based on the current cycle calibration error and the error sequence of at least three consecutive detection cycles. The candidate parameters are then jointly constrained by the low-frequency leakage, head angular velocity, and error change trend to obtain the final execution parameters. The calibration error is a weighted sum of the deviations of the low-frequency leakage of the left and right ears relative to the target low-frequency leakage, the deviations of the current left and right ear imbalance relative to the target imbalance, the comprehensive high-frequency positioning deviation, and the excess of the current audio processing delay relative to the preset delay threshold. The target low-frequency leakage, the target imbalance, and the preset delay threshold are all derived from the initialization parameter group. The final execution parameters are applied to the adaptive equalization unit, head-related transfer function fusion unit, binaural time difference correction unit, binaural sound level difference correction unit, and reflection compensation unit to process the current audio frame and obtain the left and right ear outputs. In subsequent detection cycles, the calibration error and error change are calculated based on the new left and right ear sound pressure data to correct the parameter output weights and parameter update rates in subsequent cycles. When the error continues to rise, it falls back to the previous stable parameter group or the initial parameter group to complete the adaptive calibration of immersive game audio.

[0004] Preferably, the low-energy detection signal is superimposed during normal program audio playback; sound pressure pressure data of the left ear and the right ear are acquired through near-ear microphones located 10 mm to 20 mm from the ear canal entrance on the inner side of the left and right earpieces; ambient noise and reflected sound pressure data are acquired through an in-cabin reference microphone; head yaw angle, pitch angle, roll angle and corresponding angular velocity are acquired through the inertial measurement unit integrated in the head display; and the timestamp of the head attitude data is aligned with the timestamp of the audio frame according to the same system clock.

[0005] Preferably, the low-energy detection signal is a short-duration high-frequency detection tone of 18 kHz to 22 kHz, or a short-duration sweep signal whose sound pressure level is 10 dB to 25 dB lower than the average sound pressure level of the current program audio during the detection period, and whose short-duration average sound pressure level in the corresponding frequency band is 10 dB to 25 dB lower than the short-duration average sound pressure level of the same frequency band as the current program audio; the duration of the detection signal is 20 milliseconds, and the detection period is 300 milliseconds; the sampling rate of the inertial measurement unit is 500 Hz; the initialization parameter set is the factory calibration parameter set or the first-wear calibration record parameter set.

[0006] Preferably, the left and right acoustic path transfer functions are estimated based on the detection signal and the sound pressure data of the left and right ears, respectively; based on the left and right acoustic path transfer functions, the low-frequency leakage of the left and right ears in the 80 Hz to 300 Hz frequency band, the high-frequency positioning deviation of the left and right ears in the 4 kHz to 12 kHz frequency band, and the imbalance of the left and right ears in the 500 Hz to 6 kHz frequency band are extracted. Wherein, the low-frequency leakage represents the average attenuation of the measured transfer gain relative to the reference transfer gain within the frequency band, the high-frequency positioning deviation represents the average absolute deviation of the left and right acoustic path amplitude-frequency response relative to the reference amplitude-frequency response within the frequency band, the imbalance represents the average absolute value of the sound pressure difference between the left and right ears within the frequency band, and the reference transfer gain and reference amplitude-frequency response are both derived from the initialization parameter set; the left and right ear high-frequency positioning deviations are combined into a comprehensive high-frequency positioning deviation.

[0007] Preferably, the left and right acoustic path transfer functions corresponding to five consecutive detection windows are processed by exponential moving average; frequency band data with a signal-to-noise ratio of less than six dB corresponding to the detection signal are removed; and low-frequency leakage, high-frequency positioning deviation, imbalance, early reflection, and microphone confidence are extracted based on the processed frequency band data. Wherein, the early reflection amount represents the ratio or difference between the direct sound energy within the detection window and the reflected sound energy within the preset early reflection time window, and the preset early reflection time window is a time window with a delay of one millisecond to twenty milliseconds relative to the arrival time of the direct sound; the microphone confidence amount is determined by weighted summation after normalization of the signal-to-noise ratio, distortion and continuity of the detection frequency band, and the value range is zero to one, and the continuity represents the stability of the amplitude change of the corresponding frequency band transfer function in consecutive adjacent detection windows.

[0008] Preferably, the data sequence input to the temporal deep learning model consists of input data corresponding to the most recent eight detection cycles. The multi-source data is resampled to the detection cycle time base after time alignment. The input data for each detection cycle includes low-frequency leakage of the left and right ears, high-frequency positioning deviation of the left and right ears, imbalance of the left and right ears, early reflection, ambient noise sound pressure level and frequency band noise energy, head posture data, head-related transfer function template identifier, current program loudness value, microphone confidence, and execution parameters of the previous cycle. The head-related transfer function template is identified by the number or index of a pre-stored head-related transfer function template; the previous cycle execution parameters include equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, and smoothing coefficient; the initial calibration parameters include equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, reflection compensation, parameter output weight, and parameter confidence. Wherein, the parameter output weight represents the contribution ratio of the current cycle model output parameter to the final execution parameter, the parameter confidence represents the confidence level of the model output parameter, and the value range is from zero to one; the temporal deep learning model is one of the following: recurrent neural network model, temporal convolutional network model, or transformer temporal model.

[0009] Preferably, when either the low-frequency leakage of the left ear or the low-frequency leakage of the right ear is greater than 0.45, the equalization compensation gain in the 80 Hz to 300 Hz frequency band is limited to no more than positive 4 dB; when the larger of the low-frequency leakage of the left ear and the low-frequency leakage of the right ear is greater than 0.65, low-frequency compensation is stopped and the parameter confidence level is limited to no more than 0.2. When the overall high-frequency positioning deviation is greater than eight decibels, the rate of change of the equalization compensation gain in the 4 kHz to 12 kHz frequency band is limited to no more than two decibels per hundred milliseconds, and only candidate parameters whose expected reduction in overall high-frequency positioning deviation is greater than or equal to a preset improvement threshold are retained. The preset improvement threshold is the high-frequency positioning deviation improvement threshold recorded in the initialization parameter group. When the absolute value of the head yaw angle angular velocity is greater than 120 degrees per second, the head correlation transfer function fusion ratio that most recently simultaneously satisfies the parameter confidence level being greater than 0.8 and the calibration error decreasing for two consecutive cycles is maintained. The audio buffer length is limited to within 128 sampling points, and only the binaural time difference correction, binaural sound level difference correction, and smoothing coefficient are updated. When the calibration error increases for three consecutive detection cycles, the parameter output weight, parameter update rate, reflection compensation amount and high frequency equalization compensation gain are reduced in sequence. When the error continues to rise, the parameter set is reverted to the previous stable parameter set that made the calibration error lower than the preset error threshold. The preset error threshold is derived from the initialization parameter set or the factory configuration parameters.

[0010] Preferably, the sound source signal to be rendered is sequentially subjected to equalization processing, head-related transfer function fusion, binaural time difference correction, binaural sound level difference correction, and reflection compensation according to the final execution parameters to obtain the left and right ear outputs; the final execution parameters are smoothly updated according to the candidate parameters of the current cycle and the execution parameters of the previous cycle, and the smooth update is controlled by a smoothing coefficient.

[0011] Preferably, the calibration error is determined by weighted summation after normalization of the deviation of the low-frequency leakage of the left and right ears relative to the target low-frequency leakage, the deviation of the current imbalance of the left and right ears relative to the target imbalance, the comprehensive high-frequency positioning deviation, and the excess of the current audio processing delay relative to the preset delay threshold. The target low-frequency leakage, target imbalance, and preset time delay threshold are all derived from the initialization parameter set; the error change trend is determined to be rising, falling, or stable based on the calibration error sequence of at least three consecutive detection cycles; when the current calibration error is less than the calibration error of the previous cycle and the decrease reaches the first preset threshold, the parameter output weight is increased; when the absolute value of the change in the current calibration error relative to the previous cycle is less than or equal to the second preset threshold, the parameter output weight is maintained. When the calibration error increases for three consecutive detection cycles, the calculation of the time-series deep learning model is paused, the smoothing coefficient is reduced to 0.01, and the system switches to the initialization parameter group. When the signal-to-noise ratio of the corresponding frequency band of the detection signal is greater than six dB and the low-frequency leakage of the left ear and the low-frequency leakage of the right ear are both less than 0.25, adaptive calibration is resumed. The first preset threshold and the second preset threshold are given by the initialization parameter group or the factory configuration parameters.

[0012] The deep learning-based immersive game audio adaptive tuning system includes a detection and perception module, a feature extraction module, a parameter determination module, a constraint execution module, an audio rendering execution module, and an error feedback module. The detection and perception module is used to superimpose low-energy detection signals in the audio rendering output link and acquire left and right ear sound pressure data, ambient sound pressure data, and head posture data. The feature extraction module is used to estimate the left and right acoustic path transfer functions based on the detection signal and the sound pressure data of the left and right ears, extract the ambient noise sound pressure level and frequency band noise energy based on the ambient sound pressure data, extract the loudness value of the current program based on the current program audio, and extract the low-frequency leakage of the left and right ears, the high-frequency positioning deviation of the left and right ears, the imbalance of the left and right ears, the early reflection, and the microphone confidence. The parameter determination module includes a pre-trained temporal deep learning model, which is used to generate initial calibration parameters based on the data output by the feature extraction module, environmental noise, current program loudness value, and execution parameters of the previous cycle; the constraint execution module is used to obtain the final execution parameters based on the initial calibration parameters and combined with low-frequency leakage, head angular velocity, and error change trend. The audio rendering execution module includes an adaptive equalization unit, a head-related transfer function fusion unit, a binaural time difference correction unit, a binaural sound level difference correction unit, and a reflection compensation unit, which are used to apply the final execution parameters to the audio rendering link; the error feedback module is used to recalculate the calibration error and error change based on the left and right ear sound pressure data obtained in subsequent detection cycles, and accordingly correct the parameter output weights, parameter update rates, and actions to fall back to the previous stable parameter group or initialized parameter group in subsequent cycles.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention extracts multi-dimensional acoustic features by superimposing low-energy probe signals on the audio rendering link, and inputs the temporal feature sequence into a pre-trained temporal deep learning model to generate initial tuning parameters; combined with rules such as head angular velocity and error change trend, the final execution parameters are obtained, which effectively overcomes the shortcomings of traditional methods that are difficult to synchronously correct changes in the acoustic path and are prone to parameter oscillation, and significantly improves tuning stability. 2. To address the sound field degradation caused by loose fitting, this method accurately extracts low-frequency leakage, high-frequency positioning deviation, and imbalance based on the measured acoustic path transfer function. Abnormal leakage is truncated through a constraint mechanism, and the equalization compensation gain, head-related transfer function fusion ratio, and binaural time difference and sound level difference are dynamically corrected. This effectively suppresses the increase in low-frequency leakage and the imbalance between the left and right ears, achieving high-precision restoration of the binaural sound field. 3. This invention introduces a comprehensive error feedback and state rollback mechanism; when a violent head rotation or a continuous increase in calibration error over multiple cycles is detected, the system automatically responds by limiting the change in equalization gain, freezing the reflection compensation amount, reducing the parameter update rate, or rolling back to the previous stable parameter group and the initial parameter group; maintaining the continuity of audio output parameters under complex noise fluctuation conditions ensures the stability of the audio rendering link. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating the deep learning-based adaptive audio tuning method for immersive games provided in this embodiment of the invention; Figure 2 A schematic diagram of the modules of the deep learning-based immersive game audio adaptive tuning system provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0017] like Figure 1 As shown, the deep learning-based immersive game audio adaptive tuning method includes: S1. Low-energy detection signals are superimposed on the audio rendering output link. The sound pressure levels of the left and right ears, the ambient sound pressure level, and the head posture data are obtained by the near-ear microphone, the cabin reference microphone, and the inertial measurement unit. Normal program audio data is also obtained simultaneously to extract the ambient noise sound pressure level, frequency band noise energy, and program loudness value. The initialization parameter group is used as the execution parameters in the first cycle. S2. Based on the detection signal and the sound pressure of the left and right ears, estimate the acoustic path transfer function, and obtain the instantaneous change direction of the amplitude-frequency response difference between adjacent periods. Extract the low-frequency leakage, high-frequency positioning deviation, imbalance, early reflection, and microphone confidence of the left and right ears. S3. Input the above features, head posture data, ambient noise sound pressure level, frequency band noise energy, program loudness value and the execution parameters of the previous cycle into the pre-trained temporal deep learning model, and retain the initial adjustment parameters whose compensation direction is consistent with the instantaneous change direction as candidate parameters. S4. The final execution parameters are obtained by combining low-frequency leakage data, head posture data, and candidate parameters constrained by error change trends determined based on a calibration error sequence of more than or equal to 3 consecutive cycles; wherein, the calibration error is determined based on low-frequency leakage deviation, imbalance deviation, comprehensive high-frequency positioning deviation, and time delay excess. S5. Apply the final execution parameters to the adaptive equalization, head-related transfer function fusion, binaural time difference, sound level difference correction, and reflection compensation units; subsequently, update the error based on new data to correct the parameter output weights and update rate; if the error continues to rise, revert to the previous stable or initialized parameter group.

[0018] S1 includes: A low-energy detection signal is superimposed during normal program audio playback; Sound pressure pressure data for the left and right ears are obtained through a near-ear microphone located 10 to 20 millimeters from the entrance of the ear canal on the inside of the earphone worn by the user. Ambient noise and reflected sound pressure data were acquired using an in-cabin reference microphone. The head-mounted display integrates an inertial measurement unit to obtain the head's yaw angle, pitch angle, roll angle, and corresponding angular velocity. Align the timestamps of the head pose data with the timestamps of the audio frames using the same system clock.

[0019] Low-energy detection signals include any of the following: A. Short-duration high-frequency probe tone from 18 kHz to 22 kHz; B. A short-time sweep signal whose sound pressure level is 10 to 25 dB lower than the average sound pressure level of the current program audio during the detection period, and whose short-time average sound pressure level in the corresponding frequency band is 10 to 25 dB lower than the short-time average sound pressure level of the same frequency band as the current program audio; the low-energy detection signal lasts for 20 milliseconds and has a detection period of 300 milliseconds. The sampling rate of the inertial measurement unit is 500 Hz; The initialization parameter set is the factory calibration parameter set or the first-wear calibration record parameter set.

[0020] S2 includes: The left and right acoustic path transfer functions were estimated based on the detection signals and the sound pressure data of the left and right ears, respectively. Based on the left and right acoustic path transfer functions, the low-frequency leakage of the left and right ears in the 80 Hz to 300 Hz frequency band, the high-frequency positioning deviation of the left and right ears in the 4 kHz to 12 kHz frequency band, and the imbalance of the left and right ears in the 500 Hz to 6 kHz frequency band are extracted. Among them, low-frequency leakage represents the average attenuation of the measured transfer gain relative to the reference transfer gain in the 80 Hz to 300 Hz frequency band; high-frequency positioning deviation represents the average absolute deviation of the left and right acoustic path amplitude-frequency response relative to the reference amplitude-frequency response in the 4 kHz to 12 kHz frequency band; and imbalance represents the average absolute value of the left and right ear acoustic pressure difference in the 500 Hz to 6 kHz frequency band. The reference transfer gain and reference amplitude-frequency response are both derived from the initialization parameter set. The high-frequency positioning deviations of the left and right ears are combined into a comprehensive high-frequency positioning deviation.

[0021] S2 also includes: The transfer functions of the left and right acoustic paths corresponding to five consecutive detection windows are processed by exponential moving average. Remove frequency band data with a signal-to-noise ratio below 6 dB corresponding to the detected signal; Based on the processed frequency band data, extract the low-frequency leakage of the left and right ears, the high-frequency positioning deviation of the left and right ears, the imbalance of the left and right ears, the early reflection, and the microphone confidence. Among them, the early reflection amount represents the ratio or difference between the direct sound energy in the detection window and the reflected sound energy in the preset early reflection time window. The preset early reflection time window is a time window with a delay of 1 millisecond to 20 milliseconds relative to the arrival time of the direct sound. The microphone confidence level is determined by weighted summation after normalizing the signal-to-noise ratio, distortion, and continuity of the detection band, and the value ranges from 0 to 1. Continuity represents the stability of the amplitude change of the transfer function of the corresponding frequency band in consecutive adjacent detection windows.

[0022] In this embodiment, for immersive spatial audio cabins or virtual reality experience cabins, users wear head-mounted displays and headphones with near-ear microphones. There are physical boundary reflections inside the cabin, and the user's head will continuously rotate during the interaction, causing slight changes in the wearing status of the headphones. During the data perception phase, the system superimposes a low-energy probe signal lasting 20 milliseconds onto the noise floor layer of the normal audio rendering link. To avoid interfering with the main audio content, the probe signal is selected as a short-duration high-frequency probe tone of 18 kHz to 22 kHz or a short-duration sweep signal masked by the current audio. The near-ear microphone synchronously captures the measured sound pressure data at the entrance of the ear canal, the in-cabin reference microphone extracts the environmental noise data, and the inertial measurement unit collects the head yaw angle, pitch angle, roll angle and corresponding angular velocity at a sampling rate of 500 Hz. The system performs hard alignment between the timestamp of the head posture data and the timestamp of the audio frame based on a unified hardware clock. The alignment logic is as follows: using the start timestamp of the audio frame as the reference trigger signal, the system uses nearest neighbor search in the circular buffer of the inertial measurement unit to find posture data points with an absolute time difference of no more than 1 millisecond, i.e., half a sampling period of the inertial measurement unit, as the synchronization mapping. In continuous data flow, if there is no effective matching point within the time window due to communication congestion or other reasons, or if the inertial measurement unit data is lost or delayed in a certain sampling period, the system will no longer wait. Instead, it will extract the attitude angular velocity of the previous effective period and multiply it by the difference between the current audio frame timestamp and the previous effective inertial measurement unit timestamp to perform first-order linear extrapolation interpolation. At the same time, a low confidence label will be added to the attitude data of this period to maintain the integrity of the subsequent time-series feature sequence, ensure the data continuity of the control link, and avoid system processing stagnation due to the delay of data from a single sensor. In the feature identification stage, the system uses the acquired sound pressure data and the frequency domain characterization of the known detection signal to calculate the measured transfer function of the left and right sound paths online; the system converts the time-aligned signal into a frequency domain sequence through fast Fourier transform, and uses the cross power spectrum of the acquired sound pressure signal and the detection signal divided by the autopower spectrum of the detection signal to obtain the amplitude frequency and phase frequency response of the transfer function. The system performs an exponential moving average operation in the frequency domain. For example, it updates the amplitude of the transfer function point by point using an attenuation coefficient with a current window weight of 0.3 and a historical smoothing value of 0.7, and removes data segments with a signal-to-noise ratio of less than 6 dB in the corresponding frequency band. Based on the processed transfer function, the system extracts the average attenuation of the measured transfer gain relative to the reference transfer gain in the 80 Hz to 300 Hz frequency band as the low-frequency leakage of the left and right ears; extracts the average absolute deviation of the left and right acoustic path amplitude frequency response relative to the reference amplitude frequency response in the 4 kHz to 12 kHz frequency band as the high-frequency positioning deviation of the left and right ears, and further combines the high-frequency positioning deviations of the left and right ears into a comprehensive high-frequency positioning deviation; extracts the average absolute value of the sound pressure difference between the left and right ears in the 500 Hz to 6 kHz frequency band as the imbalance of the left and right ears. Meanwhile, the system generates early reflections by separating the direct sound energy from the reflected sound energy within a delay of 1 to 20 milliseconds, and maps them to the 0 to 1 interval according to the signal-to-noise ratio, distortion, and amplitude continuity of adjacent detection windows in each detection frequency band. In the specific mapping rules, the signal-to-noise ratio mapping method is set as follows: when the signal-to-noise ratio is less than or equal to 0 dB, the mapping value is 0; when it is greater than or equal to 20 dB, the mapping value is 1; and when it is between 0 dB and 20 dB, a linear proportional mapping is performed. Distortion is calculated based on total harmonic distortion (THD). The preset maximum allowable distortion threshold is 10%. The distortion mapping value is 1 minus the ratio of the measured THD to 10%. If the measured THD is greater than 10%, the mapping value is 0. Continuity is defined as the ratio of the absolute value of the amplitude difference of the transfer function of two consecutive adjacent detection windows in the corresponding frequency bands to the current average amplitude of that frequency band. The continuity mapping value is 1 minus this ratio. If this ratio is greater than 1, the mapping value is 0. The system weights and sums the normalized mapping values ​​according to preset proportions, such as signal-to-noise ratio (SNR) 0.5, distortion 0.3, and continuity 0.2, to generate a comprehensive microphone confidence value. After the above physical features are extracted, they are written into the shared state register for subsequent decision-making links. In this embodiment, the data sequence input to the pre-trained temporal deep learning model consists of the input data corresponding to the most recent 8 probe cycles. Each input feature data is resampled to the probe cycle time base after time alignment. The input data for each detection cycle includes: low-frequency leakage in the left and right ears, high-frequency positioning deviation in the left and right ears, imbalance in the left and right ears, early reflections, ambient noise sound pressure level and frequency band noise energy, head posture data, head-related transfer function template identifier, program loudness value, microphone confidence, and execution parameters from the previous cycle. Among them, the header-related transfer function template identifier is the number or index of the pre-stored header-related transfer function template; The parameters executed in the previous cycle included equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, and smoothing coefficient; The initial calibration parameters include equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, reflection compensation, parameter output weights, and parameter reliability. Among them, the parameter output weight represents the contribution ratio of the current cycle model output parameters to the final execution parameters, and the parameter confidence represents the confidence level of the model output parameters, with a value range of 0 to 1; the pre-trained temporal deep learning model is one of the following: recurrent neural network model, temporal convolutional network model, or transformer temporal model.

[0023] S4 includes: When either the low-frequency leakage of the left ear or the low-frequency leakage of the right ear is greater than or equal to 0.45, the equalization compensation gain in the 80 Hz to 300 Hz frequency band is limited to no more than +4 dB. When the larger of the low-frequency leakage in the left ear and the low-frequency leakage in the right ear is greater than or equal to 0.65, low-frequency compensation is stopped, and the parameter confidence level is limited to no higher than 0.2. When the overall high-frequency positioning deviation is greater than or equal to 8 dB, the rate of change of the equalization compensation gain in the 4 kHz to 12 kHz band is limited to no more than 2 dB / 100 milliseconds, and only candidate parameters whose expected reduction in overall high-frequency positioning deviation is greater than or equal to the preset improvement threshold are retained. The preset improvement threshold is the high-frequency positioning deviation improvement threshold recorded in the initialization parameter group. When the absolute value of the head yaw angle angular velocity is greater than or equal to 120 degrees per second, the head correlation transfer function fusion ratio that most recently simultaneously satisfies the parameter confidence level being greater than 0.8 and the calibration error decreasing for two consecutive cycles is maintained. The audio buffer length is limited to within 128 sampling points, and only the binaural time difference correction, binaural sound level difference correction, and smoothing coefficient are updated. When the calibration error increases for three consecutive detection cycles, the parameter output weight, parameter update rate, reflection compensation amount, and high-frequency equalization compensation gain are reduced sequentially. When the error continues to rise, the parameter set is reverted to the previous stable parameter set that made the calibration error lower than the preset error threshold. The preset error threshold is derived from the initialization parameter set or the factory configuration parameters.

[0024] In S5, the sound source signal to be rendered is sequentially processed according to the final execution parameters, including equalization, head-related transfer function fusion, binaural time difference correction, binaural sound level difference correction, and reflection compensation, to obtain the left and right ear outputs. The final execution parameters are smoothly updated based on the candidate parameters of the current cycle and the execution parameters of the previous cycle, and the smooth update is controlled by a smoothing coefficient.

[0025] The low-frequency leakage deviation and imbalance deviation are deviations relative to the target value; the calibration error is determined by the fusion calculation of the low-frequency leakage deviation, imbalance deviation, comprehensive high-frequency positioning deviation, and time delay excess, and the benchmarks for each item are all from the initialization parameter set; the error change trend includes rising, falling, or stabilizing; When the current calibration error is less than the calibration error of the previous cycle and the decrease reaches the first preset threshold, increase the weight of the parameter output. When the absolute value of the change in the current calibration error relative to the previous cycle is less than the second preset threshold, the parameter output weight is maintained. When the calibration error increases for three consecutive detection cycles, the calculation of the pre-trained temporal deep learning model is paused, the smoothing coefficient is reduced to 0.01, and the initial parameter set is switched. When the signal-to-noise ratio of the corresponding frequency band of the detected signal is greater than or equal to 6 dB and the low-frequency leakage of the left ear and the low-frequency leakage of the right ear are both less than or equal to 0.25, the adaptive adjustment is restored. The first preset threshold and the second preset threshold are given by the initialization parameter group or the factory configuration parameters.

[0026] In this embodiment, the above-described spatial audio cockpit scenario continues: after acquiring multi-dimensional physical features, the system concatenates the physical features of the current cycle with the parameters executed in the previous cycle, and extracts the historical sequence of the last 8 detection cycles and inputs it into the temporal deep learning model; specifically, in terms of data organization, the system horizontally concatenates physical perception features such as low-frequency leakage in the left and right ears, and imbalance in the left and right ears with the execution parameters to form a single-cycle feature vector, and then stacks the data of the last 8 cycles along the time dimension to form a two-dimensional temporal input matrix, which is input into a model such as the self-attention layer of a converter; The exact dimension of the input feature matrix of this model is... Where 8 represents 8 consecutive detection cycles in the time dimension, and 32 represents the dimension of combining physical characteristics and execution status within a single cycle; the 32-dimensional data structure of a single cycle is rigorously defined; Its specific physical mapping is as follows: 2D low-frequency leakage of left and right ears, 2D high-frequency positioning deviation of left and right ears, 1D imbalance of left and right ears, 1D early reflection, 1D ambient noise sound pressure level, 4D frequency band noise energy divided into low, low-mid, mid-high and high frequency bands, 6D head posture data including yaw angle, pitch angle, roll angle and corresponding 3 angular velocities, 1D head correlation transfer function template identifier, 1D current program loudness value, 1D microphone confidence, and a total of 20 physical feature inputs; The 12-dimensional previous cycle execution parameter input includes equalization compensation gain for 3 frequency bands in the left and right ears (6 dimensions), 1-dimensional head correlation transfer function fusion ratio, 1-dimensional binaural time difference correction for the left and right acoustic paths (2 dimensions), 1-dimensional binaural sound level difference correction for the left and right acoustic paths (2 dimensions), and 1-dimensional smoothing coefficient. The model's network structure includes an input feature embedding layer, a multi-head self-attention module, and a feedforward network. It outputs a tuning decision vector with dimensions [1, 14] through a fully connected mapping layer. The 14-dimensional output tensor includes: 6-dimensional equalization compensation gain for 3 frequency bands on the left and right, 1-dimensional head correlation transfer function fusion ratio, 1-dimensional 2-dimensional binaural time difference correction for the left and right, 1-dimensional 2-dimensional binaural sound level difference correction for the left and right, 1-dimensional reflection compensation, 1-dimensional parameter output weights, and 1-dimensional parameter confidence. In the offline training phase, a sound field dataset containing extreme loosening and multi-level noise conditions is used to perform supervised updates to the model. The total loss function is set as the weighted sum of the mean squared error of the tuning parameter predictions and the binary cross-entropy of the parameter confidence predictions. To avoid gradient update imbalance caused by inconsistent scales, the mean squared error term is pre-normalized to... The parameters of the interval are calculated, and the specific calculation relationship is as follows: ; in, Represents the total loss function. This represents the mean square error of the calibration parameter prediction. The binary cross-entropy represents the confidence prediction of parameters. and The weighting coefficients are all fixed at 0.5; the parameter labels of the sound field dataset are composed of the optimal execution parameter matrix determined by the sound field analysis algorithm through offline traversal of global parameters to minimize the expected tuning error, and are used as the baseline label for the mean square error term. The model outputs initial calibration parameters, including equalization compensation gain and head-related transfer function fusion ratio; the system then uses the compensation gain vector of these initial calibration parameters. Instantaneous change direction vector obtained from physical identification The comparison was performed by mean-reducing the measured amplitude-frequency response difference in the frequency domain according to three equalization frequency bands for each of the left and right ears, resulting in a 6-dimensional frequency band amplitude difference vector. , Through the sign function of this vector Extract; calculate the cosine similarity of vectors using the formula: ; in, Represents the compensation gain vector With instantaneous change direction vector The angle between them; if and only if the cosine similarity is greater than 0, the compensation direction is determined to be consistent, and only the parameters in this state are retained as candidate parameters; if the model outputs nothing or the direction is completely opposite due to abnormal input data, the system discards the current model output and directly uses the execution parameters of the previous cycle as candidate parameters; During the constraint execution phase, the system does not directly transmit candidate parameters, but forcibly truncates them based on the physical state; when the low-frequency leakage of any ear is detected to be greater than or equal to 0.45, the system limits the equalization compensation gain of the 80 Hz to 300 Hz frequency band to no more than positive 4 dB. When the larger of the low-frequency leakage in the left ear and the low-frequency leakage in the right ear is greater than or equal to 0.65, the system stops low-frequency compensation and reduces parameter reliability. In addition, when the detected head yaw angle angular velocity is greater than 120° / s, the system freezes the update operation of the head-related transfer function fusion ratio, limits the audio buffer length to within 128 sampling points, and only allows the relatively small binaural time difference and sound level difference corrections to take effect. The final execution parameters, after constraint verification, are written to the control register of the audio rendering link. The rendering execution unit, based on a smoothing coefficient, smoothly transitions the target parameters with the execution parameters of the previous cycle. The specific smoothing update logic is as follows: The smoothing update formula is: ; in, Indicates the currently executed parameters. Indicates the parameters executed in the previous cycle. Indicates the target parameter. This represents the smoothing coefficient, which sequentially performs equalization, blending, correction, and reflection compensation on the current audio frame. In the specific audio control flow, the system converts the equalization compensation gain into the operating coefficients of a digital double second-order filter for time-domain correction, and fuses the head-related transfer function into a proportional gain. As a dynamic weighting factor, the target head correlation transfer function and the reference head correlation transfer function are decomposed into minimum phase response and pure delay components, which are then reconstructed after interpolation in the frequency domain and time domain, respectively, or the frequency domain magnitude logarithmic interpolation formula is used: ; The fusion calculation is performed by combining linear phase delay, where, Indicates frequency, This indicates the frequency of the fused head-related transfer function. The amplitude-frequency response at that point, This indicates the reference head related transfer function at frequency. The amplitude-frequency response at that point, This indicates the target head-related transfer function at frequency. The amplitude-frequency response at the point accurately maps the binaural time difference correction to the offset of the left and right channel delay buffer queue read pointer, maps the binaural sound level difference correction to the difference adjustment factor of the left and right channel main output gain, and applies the reflection compensation as a gain coefficient to the early environmental reflection sound rendering branch. Since a single feedforward execution cannot confirm the true improvement in the sound field, the system calculates the calibration error based on the newly acquired sound pressure data in the next detection cycle. This calibration error integrates the deviation of the low-frequency leakage of the left and right ears relative to the target low-frequency leakage, the deviation of the current imbalance of the left and right ears relative to the target imbalance, the comprehensive high-frequency positioning deviation, and the amount by which the current audio processing delay exceeds the preset delay threshold. In the specific calibration error calculation, the system divides the deviation of each feature relative to the target by its corresponding maximum allowable deviation threshold, truncates the excess portion to 1 for extreme value normalization, and then multiplies it by a preset weight and sums them up; for example, the system sets the weight of the deviation of low frequency leakage in the left and right ears to 0.4, the weight of the deviation of imbalance in the left and right ears to 0.3, the weight of the comprehensive high frequency positioning deviation to 0.2, and the weight of the delay excess to 0.1. The target low-frequency leakage and target imbalance are recorded in the initialization parameter group or set to ideal zero value by default in the initialization configuration. Assuming the current normalized value of the low-frequency leakage deviation between the left and right ears is 0.5, the current normalized value of the imbalance deviation between the left and right ears is 0.2, the normalized value of the comprehensive high-frequency positioning deviation is 0.4, and the processing delay does not exceed the threshold (i.e., the normalized value of the delay exceeding the threshold is 0), then the current calibration error is calculated as 0.5×0.4+0.2×0.3+0.4×0.2+0=0.34. If the current calibration error is less than the calibration error of the previous cycle and the decrease reaches, for example, the first preset threshold calibrated to 0.05, the system increases the parameter output weight. If the absolute value of the change in the current calibration error relative to the previous cycle is less than, for example, a second preset threshold calibrated to 0.02, the current state is determined to be stable, and the system keeps the parameter output weights unchanged; if the calibration error shows an upward trend for three consecutive cycles, the system reduces the parameter output weights and smoothing coefficients, freezes the reflection compensation amount and high-frequency equalization gain; if the error continues to worsen, the system directly pauses the deep model inference, reduces the smoothing coefficient to 0.01, and reverts to the initial parameter group; until the environmental signal-to-noise ratio is greater than 6 dB and the low-frequency leakage of the left ear and the low-frequency leakage of the right ear are both less than 0.25, the system releases the degraded state and resumes the adaptive calibration calculation of the normal cycle; like Figure 2 As shown, an immersive game audio adaptive tuning system based on deep learning, used to implement the method of the embodiment, includes: The detection and perception module is used to superimpose low-energy detection signals in the audio rendering output link and acquire left and right ear sound pressure data, ambient sound pressure data, head posture data and program audio data. The feature extraction module is used to estimate the left and right acoustic path transfer functions based on low-energy detection signals and left and right ear sound pressure data, extract the ambient noise sound pressure level and frequency band noise energy based on ambient sound pressure data, extract the program loudness value based on program audio, and extract the left and right ear low-frequency leakage, left and right ear high-frequency positioning deviation, left and right ear imbalance, early reflection, and microphone confidence. The parameter determination module includes a pre-trained temporal deep learning model, which is used to generate initial tuning parameters based on the data output by the feature extraction module, environmental noise, program loudness value, and execution parameters of the previous cycle. The constraint execution module is used to perform joint constraints based on the initial calibration parameters and combined with the low-frequency leakage, head angular velocity and error change trend to obtain the final execution parameters; The audio rendering execution module includes an adaptive equalization unit, a head-related transfer function fusion unit, a binaural time difference correction unit, a binaural sound level difference correction unit, and a reflection compensation unit, which are used to apply the final execution parameters to the audio rendering link. The error feedback module is used to recalculate the calibration error and error change based on the sound pressure data of the left and right ears obtained in subsequent detection cycles, and to correct the parameter output weight, parameter update rate, and parameter rollback to the previous stable parameter group or initial parameter group in subsequent cycles accordingly.

[0027] In this embodiment, in the system scheduling layer of the space audio cockpit, the detection and perception module serves as the underlying data source, collects raw acoustic and attitude signals according to a fixed system time base, and pushes the aligned data blocks into a shared memory queue; the feature extraction module pulls data from this queue, completes frequency domain mapping and physical feature dimensionality reduction, and updates the extracted state quantities such as low-frequency leakage of the left and right ears and high-frequency positioning deviation of the left and right ears to the global state record table. After the parameter determination module is triggered by the timer, it reads the current features and the execution parameters of the previous cycle from the global state record table, performs forward inference of the deep model, and generates preliminary decision instructions. The constraint execution module acts as a parameter state verification node, blocks abnormal preliminary decision instructions, and calls the preset rule base to forcibly truncate parameters that exceed the limit threshold, and writes the compliant final execution parameters into the hardware control register of the audio rendering execution module. The audio rendering execution module completes the processing of audio frames according to the register instructions. In the next control cycle, the error feedback module extracts the physical feedback data after rendering execution, calculates the error change trend, and writes the trend identifier back to the global state record table. This allows the control parameter determination module to determine the weight allocation and rollback actions in the next cycle, ensuring that each module maintains a stable control link operation in a decoupled state. If any module fails to complete data writing within the specified delay, the system scheduler will instruct each downstream module to read the cached data from the previous valid cycle to maintain link continuity.

[0028] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A deep learning-based adaptive audio tuning method for immersive games, characterized in that, include: S1. Low-energy detection signals are superimposed on the audio rendering output link. The sound pressure levels of the left and right ears, the ambient sound pressure level, and the head posture data are obtained by the near-ear microphone, the cabin reference microphone, and the inertial measurement unit. Normal program audio data is also obtained simultaneously to extract the ambient noise sound pressure level, frequency band noise energy, and program loudness value. The initialization parameter group is used as the execution parameters in the first cycle. S2. Based on the detection signal and the sound pressure of the left and right ears, estimate the acoustic path transfer function, and obtain the instantaneous change direction of the amplitude-frequency response difference between adjacent periods. Extract the low-frequency leakage, high-frequency positioning deviation, imbalance, early reflection, and microphone confidence of the left and right ears. S3. Input the above features, head posture data, ambient noise sound pressure level, frequency band noise energy, program loudness value and the execution parameters of the previous cycle into the pre-trained temporal deep learning model, and retain the initial adjustment parameters whose compensation direction is consistent with the instantaneous change direction as candidate parameters. S4. The final execution parameters are obtained by combining low-frequency leakage data, head posture data, and candidate parameters constrained by error change trends determined based on a calibration error sequence of more than or equal to 3 consecutive cycles; wherein, the calibration error is determined based on low-frequency leakage deviation, imbalance deviation, comprehensive high-frequency positioning deviation, and time delay excess. S5. Apply the final execution parameters to the adaptive equalization, head-related transfer function fusion, binaural time difference, sound level difference correction, and reflection compensation units; subsequently, update the error based on new data to correct the parameter output weights and update rate; if the error continues to rise, revert to the previous stable or initialized parameter group.

2. The deep learning-based immersive game audio adaptive tuning method according to claim 1, characterized in that, S1 includes: The low-energy detection signal is superimposed during normal program audio playback; The sound pressure level data of the left ear and the right ear are acquired by the near-ear microphone located 10 to 20 millimeters from the entrance of the ear canal on the inside of the earphone worn by the user; Ambient noise and reflected sound pressure data were acquired using an in-cabin reference microphone. The head-mounted display integrates an inertial measurement unit to obtain the head's yaw angle, pitch angle, roll angle, and corresponding angular velocity. The timestamps of the head pose data and the audio frame timestamps are aligned with the same system clock.

3. The deep learning-based adaptive audio tuning method for immersive games according to claim 2, characterized in that, The low-energy detection signal includes any of the following: A. Short-duration high-frequency probe tone from 18 kHz to 22 kHz; B. A short-time sweep signal whose sound pressure level is 10 to 25 dB lower than the average sound pressure level of the current program audio during the detection period, and whose short-time average sound pressure level in the corresponding frequency band is 10 to 25 dB lower than the short-time average sound pressure level of the same frequency band as the current program audio; the duration of the low-energy detection signal is 20 milliseconds, and the detection period is 300 milliseconds. The sampling rate of the inertial measurement unit is 500 Hz; The initialization parameter set is either the factory calibration parameter set or the initial wear calibration record parameter set.

4. The deep learning-based adaptive audio tuning method for immersive games according to claim 2, characterized in that, S2 includes: The left and right acoustic path transfer functions were estimated based on the detection signals and the sound pressure data of the left and right ears, respectively. Based on the left and right acoustic path transfer functions, the low-frequency leakage of the left and right ears in the 80 Hz to 300 Hz frequency band, the high-frequency positioning deviation of the left and right ears in the 4 kHz to 12 kHz frequency band, and the imbalance of the left and right ears in the 500 Hz to 6 kHz frequency band are extracted. Wherein, the low-frequency leakage represents the average attenuation of the measured transfer gain relative to the reference transfer gain in the 80 Hz to 300 Hz frequency band; the high-frequency positioning deviation represents the average absolute deviation of the left and right acoustic path amplitude-frequency response relative to the reference amplitude-frequency response in the 4 kHz to 12 kHz frequency band; the imbalance represents the average absolute value of the left and right ear acoustic pressure difference in the 500 Hz to 6 kHz frequency band; and the reference transfer gain and reference amplitude-frequency response are both derived from the initialization parameter set. The high-frequency positioning deviations of the left and right ears are combined into a comprehensive high-frequency positioning deviation.

5. The deep learning-based adaptive audio tuning method for immersive games according to claim 4, characterized in that, S2 further includes: The transfer functions of the left and right acoustic paths corresponding to five consecutive detection windows are processed by exponential moving average. Remove frequency band data with a signal-to-noise ratio below 6 dB corresponding to the detected signal; Based on the processed frequency band data, the low-frequency leakage of the left and right ears, the high-frequency positioning deviation of the left and right ears, the imbalance of the left and right ears, the early reflection, and the microphone confidence are extracted. Wherein, the early reflection amount represents the ratio or difference between the direct sound energy in the detection window and the reflected sound energy in the preset early reflection time window, and the preset early reflection time window is a time window with a delay of 1 millisecond to 20 milliseconds relative to the arrival time of the direct sound; The microphone confidence value is determined by weighted summation after normalization of the signal-to-noise ratio, distortion, and continuity of the detection frequency band, and the value ranges from 0 to 1. Continuity represents the stability of the amplitude change of the transfer function of the corresponding frequency band in consecutive adjacent detection windows.

6. The deep learning-based adaptive audio tuning method for immersive games according to claim 1, characterized in that, The data sequence input to the pre-trained temporal deep learning model consists of input data corresponding to the most recent 8 detection cycles. Each input feature data is resampled to the detection cycle time base after time alignment. The input data for each detection cycle includes: low-frequency leakage in the left and right ears, high-frequency positioning deviation in the left and right ears, imbalance in the left and right ears, early reflections, ambient noise sound pressure level and frequency band noise energy, head posture data, head-related transfer function template identifier, program loudness value, microphone confidence, and execution parameters from the previous cycle. The header-related transfer function template is identified by the number or index of a pre-stored header-related transfer function template. The parameters executed in the previous cycle include equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, and smoothing coefficient; The initial calibration parameters include equalization compensation gain, head-related transfer function fusion ratio, binaural time difference correction, binaural sound level difference correction, reflection compensation, parameter output weight, and parameter reliability. Wherein, the parameter output weight represents the contribution ratio of the current cycle model output parameter to the final execution parameter, and the parameter confidence represents the confidence level of the model output parameter, with a value range of 0 to 1; the pre-trained temporal deep learning model is one of the following: recurrent neural network model, temporal convolutional network model, or transformer temporal model.

7. The deep learning-based immersive game audio adaptive tuning method according to claim 6, characterized in that, S4 includes: When either the low-frequency leakage of the left ear or the low-frequency leakage of the right ear is greater than or equal to 0.45, the equalization compensation gain in the 80 Hz to 300 Hz frequency band is limited to no more than +4 dB. When the larger of the low-frequency leakage in the left ear and the low-frequency leakage in the right ear is greater than or equal to 0.65, low-frequency compensation is stopped, and the parameter confidence level is limited to no higher than 0.

2. When the overall high-frequency positioning deviation is greater than or equal to 8 dB, the rate of change of the equalization compensation gain in the 4 kHz to 12 kHz band is limited to no more than 2 dB / 100 milliseconds, and only candidate parameters whose expected reduction in overall high-frequency positioning deviation is greater than or equal to the preset improvement threshold are retained. The preset improvement threshold is the high-frequency positioning deviation improvement threshold recorded in the initialization parameter group. When the absolute value of the head yaw angle angular velocity is greater than or equal to 120 degrees per second, the head correlation transfer function fusion ratio that most recently simultaneously satisfies the parameter confidence level being greater than 0.8 and the calibration error decreasing for two consecutive cycles is maintained. The audio buffer length is limited to within 128 sampling points, and only the binaural time difference correction, binaural sound level difference correction, and smoothing coefficient are updated. When the calibration error increases for three consecutive detection cycles, the parameter output weight, parameter update rate, reflection compensation amount, and high-frequency equalization compensation gain are reduced sequentially. When the error continues to rise, the parameter set is reverted to the previous stable parameter set that made the calibration error lower than the preset error threshold. The preset error threshold is derived from the initialization parameter set or the factory configuration parameters.

8. The deep learning-based immersive game audio adaptive tuning method according to claim 6, characterized in that, In S5, the sound source signal to be rendered is sequentially subjected to equalization processing, head-related transfer function fusion, binaural time difference correction, binaural sound level difference correction and reflection compensation according to the final execution parameters to obtain the left and right ear outputs. The final execution parameters are smoothly updated according to the candidate parameters of the current cycle and the execution parameters of the previous cycle, and the smooth update is controlled by a smoothing coefficient.

9. The deep learning-based immersive game audio adaptive tuning method according to claim 8, characterized in that, The low-frequency leakage deviation and the imbalance deviation are deviations relative to the target value; the calibration error is determined based on the fusion calculation of the low-frequency leakage deviation, the imbalance deviation, the comprehensive high-frequency positioning deviation, and the time delay excess, and the benchmarks for each item are all from the initialization parameter group; the error change trend includes rising, falling, or stabilizing; When the current calibration error is less than the calibration error of the previous cycle and the decrease reaches the first preset threshold, increase the weight of the parameter output. When the absolute value of the change in the current calibration error relative to the previous cycle is less than the second preset threshold, the parameter output weight is maintained. When the calibration error increases for three consecutive detection cycles, the calculation of the pre-trained temporal deep learning model is paused, the smoothing coefficient is reduced to 0.01, and the system is switched to the initial parameter set. When the signal-to-noise ratio of the corresponding frequency band of the detected signal is greater than or equal to 6 dB and the low-frequency leakage of the left ear and the low-frequency leakage of the right ear are both less than or equal to 0.25, the adaptive adjustment is restored. The first preset threshold and the second preset threshold are given by the initialization parameter group or the factory configuration parameters.

10. A deep learning-based immersive game audio adaptive tuning system, used to implement the method described in any one of claims 1 to 9, characterized in that, include: The detection and perception module is used to superimpose low-energy detection signals in the audio rendering output link and acquire left and right ear sound pressure data, ambient sound pressure data, head posture data and program audio data. The feature extraction module is used to estimate the left and right acoustic path transfer functions based on the low-energy detection signal and the left and right ear sound pressure data, extract the ambient noise sound pressure level and frequency band noise energy based on the ambient sound pressure data, extract the program loudness value based on the program audio, and extract the left and right ear low-frequency leakage, left and right ear high-frequency positioning deviation, left and right ear imbalance, early reflection, and microphone confidence. The parameter determination module includes a pre-trained temporal deep learning model, which is used to generate initial tuning parameters based on the data output by the feature extraction module, environmental noise, program loudness value, and execution parameters of the previous cycle. The constraint execution module is used to perform joint constraints based on the initial calibration parameters and in combination with low-frequency leakage, head angular velocity and error change trends to obtain the final execution parameters; The audio rendering execution module includes an adaptive equalization unit, a head-related transfer function fusion unit, a binaural time difference correction unit, a binaural sound level difference correction unit, and a reflection compensation unit, which are used to apply the final execution parameters to the audio rendering link. The error feedback module is used to recalculate the calibration error and error change based on the sound pressure data of the left and right ears obtained in subsequent detection cycles, and to correct the parameter output weight, parameter update rate, and parameter rollback to the previous stable parameter group or initial parameter group in subsequent cycles accordingly.

Images