Method and system for immersive in-vehicle sound generation with multi-channel collaboration processing

Abstract

The application discloses an immersive vehicle sound effect generation method and system based on multi-channel cooperative processing, and relates to the technical field of vehicle sound effects. The method comprises the following steps: obtaining a plurality of audio channels; performing multi-channel cooperative processing on the plurality of audio channels to obtain a first sequence of channel signals; performing immersive optimization on the first sequence of channel signals to obtain a second sequence of channel signals; performing signal interference compensation on the second sequence of channel signals to obtain a third sequence of channel signals; performing adaptive sound effect enhancement on the third sequence of channel signals to obtain a fourth sequence of channel signals; transmitting the fourth sequence of channel signals to the plurality of audio channels, collecting actual sound field feedback information, and performing sound effect feedback optimization on the plurality of audio channels according to the actual sound field feedback information. The technical problem that the vehicle sound effect lacks immersion due to changes in the dynamic environment inside and outside the vehicle in the prior art is solved, and the technical effect of improving the immersion of the vehicle sound effect is achieved through multi-channel cooperative processing.

Description

Technical Field

[0001] The present invention relates to the field of in-vehicle sound effect technology, and in particular to a method and system for generating immersive in-vehicle sound effects using multi-channel collaborative processing. Background Art

[0002] With the continuous development of intelligent automotive technology, in-car entertainment systems have become an important component in enhancing the driving experience. The quality of in-car sound effects directly affects the audio experience of drivers and passengers, especially in the application of multi-channel sound effects and immersive sound effects, which are in increasing demand. However, existing in-car sound effect systems often face the following problems: the dynamic changes in the internal and external environments of the car, such as vehicle speed, noise, and interior layout, make it difficult to stably optimize the sound effects, often resulting in sound quality distortion and lack of immersion. At the same time, traditional sound effect processing methods fail to fully consider the special characteristics of multi-channel collaborative processing and the interior space of the car, making it difficult to provide car owners with a high-quality audio experience. Summary of the Invention

[0003] The present application provides a method and system for generating immersive in-vehicle sound effects using multi-channel collaborative processing, which solves the technical problem in the prior art of lack of immersion in in-vehicle sound effects due to dynamic environmental changes inside and outside the vehicle.

[0004] In a first aspect of the present application, a method for generating immersive in-vehicle sound effects using multi-channel collaborative processing is provided, the method comprising:

[0005] Multiple car audio systems are arranged in a target car to obtain multiple audio channels; multi-channel collaborative processing is performed on the multiple audio channels according to a sound source signal to obtain a first sequence of channel signals; immersion optimization is performed on the first sequence of channel signals according to the in-vehicle sound effect focusing factor of the target car to obtain a second sequence of channel signals; based on a layout-affected sound effect compensation function, signal interference compensation is performed on the second sequence of channel signals according to the in-vehicle space layout model of the target car to obtain a third sequence of channel signals; adaptive sound effect enhancement is performed on the third sequence of channel signals according to the dynamic environment field inside and outside the target car to obtain a fourth sequence of channel signals; the fourth sequence of channel signals is transmitted to the multiple audio channels, actual sound field feedback information is collected through an in-vehicle microphone array, and sound effect feedback optimization is performed on the multiple audio channels according to the actual sound field feedback information.

[0006] A second aspect of the present application provides an immersive in-vehicle sound effect generation system for multi-channel collaborative processing, the system comprising:

[0007] An audio channel acquisition module is used to set up multiple car audio systems in a target car to obtain multiple audio channels; a collaborative processing module is used to perform multi-channel collaborative processing on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals; an immersive optimization module is used to perform immersive optimization on the first sequence of channel signals according to the in-vehicle sound effect focusing factor of the target car to obtain a second sequence of channel signals; a sound effect compensation module is used to perform signal interference compensation on the second sequence of channel signals according to the in-vehicle space layout model of the target car based on a layout-affected sound effect compensation function to obtain a third sequence of channel signals; a sound effect enhancement module is used to perform adaptive sound effect enhancement on the third sequence of channel signals according to the dynamic environment field inside and outside the target car to obtain a fourth sequence of channel signals; a feedback optimization module is used to transmit the fourth sequence of channel signals to the multiple audio channels, collect actual sound field feedback information through the in-vehicle microphone array, and perform sound effect feedback optimization on the multiple audio channels according to the actual sound field feedback information.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] First, multiple car audio systems are installed in a target vehicle to obtain multiple audio channels. Next, multi-channel collaborative processing is performed on the multiple audio channels based on the sound source signal to obtain a first sequence of channel signals. The first sequence of channel signals is further immersively optimized based on the target vehicle's in-vehicle sound effect focus factor to obtain a second sequence of channel signals. Then, based on a layout-affected sound effect compensation function, signal interference compensation is performed on the second sequence of channel signals according to the target vehicle's interior spatial layout model to obtain a third sequence of channel signals. Next, adaptive sound effect enhancement is performed on the third sequence of channel signals based on the target vehicle's in-vehicle and in-vehicle dynamic environment to obtain a fourth sequence of channel signals. Finally, the fourth sequence of channel signals is transmitted to multiple audio channels, and actual sound field feedback information is collected via an in-vehicle microphone array. Based on this actual sound field feedback, sound effect feedback optimization is performed on the multiple audio channels. This solves the technical problem in the prior art of lacking immersive in-vehicle sound effects due to dynamic changes in the vehicle's in-vehicle and in-vehicle environment. By using multi-channel collaborative processing, the immersiveness of in-vehicle sound effects is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1A schematic flow chart of a method for generating immersive in-vehicle sound effects using multi-channel collaborative processing provided in an embodiment of the present application;

[0012] Figure 2 Schematic diagram of the structure of the immersive in-vehicle sound generation system for multi-channel collaborative processing provided in an embodiment of the present application.

[0013] Explanation of the reference numerals: audio channel acquisition module 11, collaborative processing module 12, immersive optimization module 13, sound effect compensation module 14, sound effect enhancement module 15, feedback optimization module 16. DETAILED DESCRIPTION

[0014] This application solves the technical problem in the prior art that in-vehicle sound effects lack immersion due to dynamic environmental changes inside and outside the vehicle by providing an immersive in-vehicle sound effect generation method and system with multi-channel collaborative processing.

[0015] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0016] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products or devices.

[0017] Example 1, as Figure 1 As shown, the present application provides a method for generating immersive in-vehicle sound effects using multi-channel collaborative processing, wherein the method includes:

[0018] Set up multiple car stereos in the target car to obtain multiple audio channels.

[0019] Based on the target vehicle's interior layout and audio design requirements, multiple high-quality car audio systems are installed in various locations within the vehicle. For example, speakers can be placed at the front and rear of the vehicle, in the doors, and on the roof, ensuring comprehensive sound coverage within the vehicle. The car audio system's controller transmits the audio source signal to each speaker, and the audio signal received by each speaker corresponds to an independent audio channel, generating an audio signal stream for each speaker. In the car audio system, the audio signal received by each speaker is defined as an independent audio channel. For example, if a car has multiple speakers, such as front speakers, rear speakers, and door speakers, each speaker corresponds to an independent audio channel, thus forming a multi-channel audio system.

[0020] Multi-channel collaborative processing is performed on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals.

[0021] A sound source signal is received from an in-vehicle audio system or an external audio source, and denoising is performed on the signal to remove interference noise and improve signal quality; multi-channel collaborative processing is performed on the processed enhanced sound source signal to generate a first sequence of channel signals.

[0022] Furthermore, performing multi-channel collaborative processing on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals includes:

[0023] De-noising is performed on the sound source signal to obtain an enhanced sound source signal; response characteristics of the multiple audio channels are tested based on the enhanced sound source signal to establish a multi-channel response characteristic matrix; and multi-channel collaborative processing is performed on the enhanced sound source signal based on the multi-channel response characteristic matrix to generate the first sequence of channel signals.

[0024] First, the audio source signal is received and denoised. Denoising removes interfering noise from the audio source signal through a filter or noise suppression algorithm, thereby obtaining an enhanced audio source signal. The enhanced audio source signal has higher signal quality and provides a clear and accurate input signal for subsequent audio processing. Next, the enhanced audio source signal is used to test the response characteristics of multiple audio channels, where the response characteristics of each audio channel include frequency response, phase response, and delay response. By testing each audio channel, the response data of each channel is obtained, and based on this data, a multi-channel response characteristic matrix is ​​established. The multi-channel response characteristic matrix is ​​used to characterize the differences in the response of each audio channel to the audio source signal.

[0025] After obtaining the multi-channel response characteristic matrix, this matrix is ​​used to perform multi-channel collaborative processing on the enhanced sound source signal. Specifically, multi-channel collaborative processing weights, adjusts the time delay, and adjusts the amplitude of the signal of each audio channel to ensure that the output signal of each audio channel can be spatially coordinated with each other and maintain the balance of the sound effect. After multi-channel collaborative processing, the optimized first sequence of channel signals is finally generated. The first sequence of channel signals will serve as the basis for subsequent sound effect optimization and enhancement, ensuring that the in-vehicle sound system can provide an immersive and high-quality audio experience.

[0026] Furthermore, performing response characteristic tests on the plurality of audio channels according to the enhanced sound source signal to establish a multi-channel response characteristic matrix includes:

[0027] A frequency response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel frequency response eigenvector; a phase response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel phase response eigenvector; a delay response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel delay response eigenvector; and the multi-channel response characteristic matrix is ​​constructed based on the multi-channel frequency response eigenvector, the multi-channel phase response eigenvector and the multi-channel delay response eigenvector.

[0028] Frequency response confidence testing is performed on multiple audio channels using an enhanced sound source signal. This test measures the response characteristics of each audio channel to audio signals within different frequency ranges. Specifically, during the test, the frequency components of the audio signal are gradually adjusted, and the gain and attenuation of each channel for signals of different frequencies are analyzed to obtain the frequency response characteristics of each audio channel. The frequency response characteristic data of each channel is organized into a multi-channel frequency response feature vector, which reflects the performance of each audio channel at different frequencies.

[0029] Phase response confidence testing is performed on multiple audio channels using enhanced source signals. Phase variations in each audio channel are analyzed to assess phase differences between channels. Specifically, the phase shift of the signal output from each audio channel is measured to obtain phase response characteristics for each audio channel. These characteristics help determine signal synchronization and phase differences between the channels. The resulting phase response characteristics form a multi-channel phase response feature vector, reflecting the time delay characteristics of the signals in each channel.

[0030] The delay response confidence test is performed on multiple audio channels using enhanced sound source signals. The response time of each audio channel is tested to identify the signal transmission delay between different channels. The test results constitute a multi-channel delay response feature vector, which reflects the delay characteristics of each channel signal.

[0031] According to the obtained multi-channel frequency response feature vector, multi-channel phase response feature vector and multi-channel time delay response feature vector, a multi-channel response characteristic matrix is constructed, which integrates all response characteristics of multiple audio channels in frequency, phase and time delay, and comprehensively characterizes the performance characteristics of each audio channel, providing comprehensive response data support for subsequent multi-channel cooperative processing.

[0032] According to the vehicle sound effect focusing factor of the target vehicle, the first sequence of channel signals is immersive optimized to obtain a second sequence of channel signals.

[0033] The vehicle sound effect focusing factor can be set according to different in-vehicle sound effect requirements, including but not limited to a characteristic user priority mode and a whole vehicle equalization mode. The characteristic user priority mode refers to optimization according to the sound effect preference of a specific in-vehicle user; and the whole vehicle equalization mode refers to comprehensive adjustment according to the sound effect preferences of all in-vehicle users to ensure the sound effect balance of each area in the vehicle. Based on the vehicle sound effect focusing factor, the first sequence of channel signals is immersive optimized to obtain the second sequence of channel signals.

[0034] Further, according to the vehicle sound effect focusing factor of the target vehicle, the first sequence of channel signals is immersive optimized to obtain a second sequence of channel signals, including:

[0035] When the vehicle sound effect focusing factor is a characteristic user priority, a first vehicle sound effect preference portrait is constructed according to the vehicle sound effect preference information of the characteristic user; when the vehicle sound effect focusing factor is a whole vehicle equalization, a second vehicle sound effect preference portrait is constructed by data fusion according to multiple in-vehicle user sound effect preference data; and the first vehicle sound effect preference portrait or the second vehicle sound effect preference portrait is used to adaptively and immersive optimize the first sequence of channel signals to generate the second sequence of channel signals.

[0036] When the in-vehicle sound effect focus factor prioritizes a characteristic user, the system first constructs a first in-vehicle sound effect preference profile based on the characteristic user's in-vehicle sound effect preference information. This in-vehicle sound effect preference information includes the characteristic user's different sound effect requirements, such as low-frequency enhancement, high-frequency suppression, or a specific volume setting. By analyzing the in-vehicle sound effect preference information, a first in-vehicle sound effect preference profile is generated that reflects the characteristic user's sound effect requirements. The system then conducts in-depth analysis of the first in-vehicle sound effect preference profile to identify the user's primary sound effect requirements. Based on the first in-vehicle sound effect preference profile, the system determines which frequency bands need to be enhanced or reduced, or which sound quality characteristics (such as clarity and bass depth) need to be optimized. Based on the first in-vehicle sound effect preference profile, the system adaptively optimizes the first sequence of channel signals for immersion to meet the characteristic user's sound effect requirements, generating a second sequence of optimized channel signals. Specifically, the system adjusts the gain of the channel signals in different frequency bands based on the characteristic user's frequency preferences to enhance or reduce specific frequency bands. For example, for a user who prefers low frequencies, the system enhances the strength of the low-frequency signal; for a user who prefers high frequencies, the system enhances the high-frequency representation. Based on the user's sound quality requirements, the system can adjust the volume, balance, and spatial sense of each channel to make the sound clearer and more natural. According to the spatial layout of the car and the user's preferences, the sound positioning of each channel is optimized to make the sound more immersive and surround.

[0037] When the in-vehicle sound focus factor is set to full vehicle balance, the system collects sound preference data from multiple in-vehicle users. This data comes from passengers in different seats in the vehicle and covers their respective requirements for volume, frequency response, sound quality, and other aspects. The system uses a data fusion algorithm to comprehensively analyze this preference data and construct a second in-vehicle sound preference profile, which reflects the common sound needs and preferences of all passengers in the vehicle. Based on this second in-vehicle sound preference profile, the system performs adaptive immersion optimization on the first sequence of channel signals to ensure balanced and coordinated sound output in every area of ​​the vehicle, thereby generating the second sequence of channel signals.

[0038] Based on the layout-affected sound effect compensation function, signal interference compensation is performed on the second sequence of channel signals according to the interior space layout model of the target vehicle to obtain a third sequence of channel signals.

[0039] Based on the interior spatial layout model, the vehicle's geometric structure and acoustic characteristics are established, identifying potential interference sources such as seats, windows, and interior materials. Then, using the layout-impacted sound compensation function, the impact of these interference sources on each channel is quantified and the corresponding compensation value is calculated. This function adjusts the channel signal's amplitude, phase, or frequency response based on the interference source's compensation weight and the interference value, eliminating the negative impact of the interior spatial layout and interference sources on the sound quality. Finally, after compensation, an optimized third sequence of channel signals is generated, ensuring a purer and more accurate sound output.

[0040] Further, the layout-influenced sound effect compensation function is:

[0041] ; wherein, CSL2 represents any channel signal in the second sequence of channel signals, CSL3 represents the channel signal after interference compensation corresponding to CSL2, J represents the number of in-vehicle interference sources of the target vehicle, L space,j represents the predicted interference value of the jth interference source to CSL2, j and J are positive integers, 1≤j≤J, C j represents L space,j corresponding compensation weight coefficient, β represents error correction coefficient.

[0042] Through the layout-influenced sound effect compensation function system, the influence value of each interference source on the channel signal can be calculated according to the influence of different in-vehicle interference sources, and the amplitude of the signal can be adjusted through the compensation weight coefficient, so as to optimize the channel signal, reduce or eliminate the negative influence of in-vehicle environmental interference on the sound effect, and finally generate the optimized channel signal CSL3.

[0043] Further, based on the layout-influenced sound effect compensation function, signal interference compensation is performed on the second sequence of channel signals according to the in-vehicle space layout model of the target vehicle to obtain a third sequence of channel signals, including:

[0044] According to the in-vehicle space layout model, sound effect interference source identification is performed to determine a plurality of interference sources; any channel signal in the second sequence of channel signals is taken as a first channel signal; interference prediction is performed on the first channel signal according to the plurality of interference sources to determine a first interference prediction sequence; proportion calculation is performed according to the first interference prediction sequence to obtain a first compensation weight sequence; based on the first interference prediction sequence and the first compensation weight sequence, signal interference compensation is performed on the first channel signal according to the layout-influenced sound effect compensation function to obtain the third sequence of channel signals.

[0045] Firstly, according to the in-vehicle space layout model, the system analyzes the geometric structure and acoustic characteristics of each part of the vehicle interior, and identifies a plurality of possible sound effect interference sources, which can be reflective surfaces such as seats, windows, roofs, etc. in the vehicle, or other factors that can affect the propagation of sound effects. Next, the system selects any channel signal from the second sequence of channel signals as the first channel signal for processing; according to the identified plurality of interference sources, the system performs interference prediction on the first channel signal; through the analysis of each interference source, the system calculates the influence of these interference sources on the first channel signal, generates a first interference prediction sequence, and the first interference prediction sequence includes the predicted interference value of each interference source to the first channel signal; according to the first interference prediction sequence, the system performs proportion calculation to obtain a first compensation weight sequence, and the first compensation weight sequence includes the compensation weight coefficient corresponding to each predicted interference value in the first interference prediction sequence. Finally, based on the first interference prediction sequence and the first compensation weight sequence, the system uses a layout-influenced sound effect compensation function to perform signal interference compensation on the first channel signal. Through the compensation function, the system adjusts the first channel signal to reduce or eliminate the influence of the interference source, thereby obtaining an optimized third sequence of channel signals.

[0046] Further, according to the plurality of interference sources, the first channel signal is predicted to interfere, and a first interference prediction sequence is determined, which includes:

[0047] According to the in-vehicle space layout model, the plurality of interference sources are characterized, and the position characteristics, structure characteristics and material characteristics of each interference source are obtained; according to the channel signal interference record set, P learner is supervised training, and P channel signal interference prediction model is obtained, P is a positive integer greater than 1; the first channel signal, the position characteristics of each interference source, the structure characteristics of each interference source and the material characteristics of each interference source are input into the P channel signal interference prediction model, and a plurality of signal interference prediction sets corresponding to the plurality of interference sources are obtained; the plurality of signal interference prediction sets are respectively calculated to obtain the first interference prediction sequence.

[0048] First, based on the interior spatial layout model, the system identifies the characteristics of multiple interference sources. Each interference source may originate from different areas within the vehicle, such as reflective surfaces like seats, windows, and doors, or sound-absorbing materials. By analyzing factors such as the interior geometry, seat position, and material structure, the system obtains the location, structure, and material characteristics of each interference source. Location characteristics refer to the spatial location of the interference source within the vehicle, including information such as distance, angle, and orientation. Structural characteristics refer to the geometry or surface structure of the interference source, such as the shape of reflective surfaces or soft and hard materials. Material characteristics refer to the material properties at the location of the interference source, such as sound absorption, reflection, or conduction. Next, the system uses a set of recorded acoustic channel signal interference records for supervised training of P learners, where P is a positive integer greater than 1. By learning from historical data, each learner is able to build an acoustic channel signal interference prediction model for each interference source. This model, based on the interference source's characteristics (including location, structure, and material characteristics) and the actual recorded interference data, predicts the specific impact of the interference source on the acoustic channel signal. The first channel signal and the positional features, structural features, and material features of each interference source are input into the trained P channel signal interference prediction models; each model performs calculations based on the input data to obtain the interference prediction results of each interference source on the first channel signal. Ultimately, the system will obtain multiple signal interference prediction sets corresponding to multiple interference sources, and each signal interference prediction set includes P channel signal interference prediction values ​​corresponding to each interference source. Finally, the centralized value calculation (for example, taking the average or median) is performed on each of the multiple signal interference prediction sets obtained. Through this calculation, the system obtains a unified interference prediction sequence, namely the first interference prediction sequence, which includes the comprehensive impact of all interference sources on the first channel signal and provides accurate interference compensation parameters for subsequent compensation and optimization processes.

[0049] Adaptively enhance the sound effects of the third sequence of channel signals according to the dynamic environment field inside and outside the target vehicle to obtain a fourth sequence of channel signals.

[0050] First, the system analyzes the dynamic environmental fields inside and outside the vehicle, such as vehicle speed, interior noise, and exterior road noise, and monitors environmental changes in real time. The system then performs a sound loss source analysis on the third sequence of the sound channel signal, evaluating the impact of the interior and exterior environments on the sound effects and generating a sound loss source factor. Next, the sound loss factors inside and outside the vehicle are integrated to generate a third sound loss source factor. Finally, based on this factor, the system performs adaptive sound enhancement on the sound channel signal, optimizing parameters such as frequency, amplitude, and delay to compensate for environmental influences, ultimately generating a fourth sequence of the sound channel signal.

[0051] Furthermore, adaptively enhancing the sound effects of the third sequence of channel signals according to the dynamic environment field inside and outside the target vehicle to obtain a fourth sequence of channel signals includes:

[0052] The dynamic environment fields inside and outside the vehicle include the dynamic environment field inside the vehicle and the dynamic environment field outside the vehicle; the sound effect loss of the third sequence of the channel signal is traced according to the dynamic environment field inside the vehicle to obtain a first sound effect loss tracing factor; the sound effect loss of the third sequence of the channel signal is traced according to the dynamic environment field outside the vehicle to obtain a second sound effect loss tracing factor; the first sound effect loss tracing factor and the second sound effect loss tracing factor are integrated to obtain a third sound effect loss tracing factor; the sound effect of the third sequence of the channel signal is enhanced according to the third sound effect loss tracing factor to generate the fourth sequence of the channel signal.

[0053] The dynamic environmental fields inside and outside the vehicle include the in-vehicle dynamic environmental field and the external dynamic environmental field. The in-vehicle dynamic environmental field mainly involves various factors inside the vehicle, such as vehicle speed changes, interior noise (such as engine noise and tire noise), seat position adjustment, and the operating status of equipment such as air conditioning, all of which will affect the performance of the in-vehicle sound effects. The external dynamic environmental field includes environmental factors such as road noise, external traffic sounds, and weather changes, all of which can also affect the in-vehicle sound effects through windows and other openings. The system collects this dynamic environmental data through real-time sensors and environmental monitoring equipment (such as vehicle speed sensors and noise sensors).

[0054] The system performs a sound effect loss tracing analysis on the third sequence of the sound channel signal based on the dynamic environment field inside the car. Specifically, the system analyzes the impact of factors such as in-car noise, seat adjustment, and vehicle speed changes on the third sequence of the sound channel signal, and generates a first sound effect loss tracing factor. This factor reflects the specific impact of the in-car environment on the sound effect, such as the loss of low frequencies caused by increased vehicle speed, or the confusion of sound effects caused by in-car noise. Then, the system performs a sound effect loss tracing analysis on the third sequence of the sound channel signal based on the dynamic environment field outside the car, and generates a second sound effect loss tracing factor. Subsequently, the system fuses the first sound effect loss tracing factor and the second sound effect loss tracing factor obtained from the analysis of the dynamic environment fields inside and outside the car to obtain a comprehensive sound effect loss tracing third factor. This factor integrates the comprehensive impact of all environmental factors inside and outside the car on the in-car sound effect, providing a basic basis for subsequent sound effect enhancement. Finally, based on the third sound effect loss tracing factor, the system performs adaptive sound effect enhancement on the third sequence of the sound channel signal. The system adjusts channel signal parameters such as frequency, amplitude, and latency to compensate for the negative impact of the dynamic environment inside and outside the vehicle on the sound quality, optimizing sound performance for a clearer, more balanced sound quality that adapts to changes in the vehicle's interior and exterior environments. After sound enhancement, the resulting optimized signal is the fourth channel signal sequence. This sequence represents the final sound signal after sound enhancement processing, under the influence of changes in the vehicle's interior and exterior dynamic environment. This sequence provides a higher-quality audio experience, ensuring that the in-vehicle sound quality always maintains optimal performance.

[0055] Furthermore, tracing the source of sound loss of the third sequence of the sound channel signal according to the dynamic environment field in the vehicle to obtain a first factor of tracing the source of sound loss includes:

[0056] According to the dynamic environment field inside the vehicle, sound effect loss is predicted for each channel signal in the third sequence of the channel signal to obtain multiple signal sound effect loss coefficients; it is determined whether the multiple signal sound effect loss coefficients are greater than or equal to a sound effect loss threshold; if the multiple signal sound effect loss coefficients are greater than or equal to the sound effect loss threshold, the loss abnormal channel signal corresponding to the abnormal sound effect loss coefficient is determined; according to the loss abnormal channel signal, the dynamic environment field inside the vehicle is correlated and traced to obtain the first sound effect loss tracing factor.

[0057] First, the system predicts the sound loss of each channel signal within the third sequence based on the dynamic in-car environment. By analyzing various factors within the in-car environment (such as vehicle speed, interior noise, and seat position changes), it predicts the degree of sound loss for each channel signal due to these environmental changes, generating multiple signal sound loss coefficients that represent the degree of impact of these environmental changes on the sound quality. Next, the system evaluates these signal sound loss coefficients, comparing them to see if they are greater than or equal to a set sound loss threshold. This threshold is a preset standard; if a loss coefficient is greater than or equal to this threshold, the channel signal is deemed to have exceeded an acceptable sound loss range and requires special processing. This means the signal sound loss coefficient is marked as abnormal.

[0058] If some values ​​among the multiple signal sound loss coefficients are greater than or equal to the sound loss threshold, the system will determine the abnormal loss channel signals corresponding to these abnormal loss coefficients. The abnormal loss channel signals indicate that the sound effects of certain channels are severely damaged under changes in the in-car environment, which may be caused by factors such as in-car noise, seat position, and changes in vehicle speed, and require key adjustments and optimizations. Based on the abnormal loss channel signals, the system correlates and traces the dynamic environmental field in the car, analyzes and determines the correlation between these abnormal loss signals and the in-car environment. For example, the system may use factors such as vehicle speed, seat position, and in-car noise to trace which environmental features are most likely to cause sound loss. Through this tracing analysis, the system can derive the first factor of sound loss tracing, which reflects the specific impact of the dynamic environmental field in the car on sound loss.

[0059] Furthermore, the system performs sound loss tracing on the third sequence of channel signals based on the external dynamic environmental field, obtaining the second factor for sound loss tracing in a similar process to the first factor. Specifically, the system analyzes various factors in the external dynamic environmental field, such as road noise, external traffic sounds, wind noise, and weather changes. These external environmental factors can enter the vehicle through windows, doors, and other locations, interfering with the sound quality inside the vehicle. Based on these external factors, the system predicts the degree of sound loss they will cause on the third sequence of channel signals. The system performs sound loss prediction on each channel signal and calculates a sound loss coefficient for each channel signal. These coefficients represent the degree to which changes in the external environment affect the sound quality. Similar to the analysis of the in-vehicle environment, the system quantifies the sound loss of each channel signal based on factors such as road noise and wind noise, and uses this as the second factor for sound loss tracing. Next, the system determines whether these sound loss coefficients are greater than or equal to a set sound loss threshold. If any loss coefficient exceeds the threshold, the system determines that these channel signals have abnormal sound loss. Then, through further correlation and tracing with the dynamic environmental field outside the vehicle, the system analyzes the relationship between these abnormal loss signals and the environmental factors outside the vehicle, thereby deriving the second factor for tracing the source of sound loss.

[0060] The fourth sequence of channel signals is transmitted to the multiple audio channels, actual sound field feedback information is collected through the in-vehicle microphone array, and sound effect feedback optimization is performed on the multiple audio channels based on the actual sound field feedback information.

[0061] The system transmits the resulting fourth sequence of channel signals, after multi-channel collaborative processing, immersion optimization, interference compensation, and adaptive sound enhancement, to the target vehicle's multiple audio channels, controlling the corresponding onboard speakers for sound output. As the channel signals play within the vehicle, the system uses a microphone array positioned at various locations within the vehicle to capture the actual sound field in real time. The microphone array's distribution is optimized to cover typical listening areas, such as the driver's seat, passenger seat, and rear seats, to capture a realistic sound field representation throughout the vehicle. The system analyzes the collected actual sound field data to extract key acoustic characteristics, including actual sound pressure level (SPL), frequency response, phase characteristics, sound field balance, and spatial sound image localization deviation. These actual sound field characteristics are then compared with a pre-set ideal sound field model to identify discrepancies between the sound output and the target effect. If deviations are detected in the actual output, such as over-attenuation in certain frequency bands, sound image localization deviation, or localized sound pressure unevenness, the system triggers a feedback optimization mechanism based on these deviations. During the sound effect feedback optimization process, the system dynamically adjusts the output parameters of each audio channel based on the difference information, including but not limited to volume gain, frequency gain, phase delay, time domain correction, etc., thereby optimizing the fourth sequence of the channel signal in real time.

[0062] In summary, the embodiments of the present application have at least the following technical effects:

[0063] First, multiple car audio systems are installed in a target vehicle to obtain multiple audio channels. Next, multi-channel collaborative processing is performed on the multiple audio channels based on the sound source signal to obtain a first sequence of channel signals. The first sequence of channel signals is further immersively optimized based on the target vehicle's in-vehicle sound effect focus factor to obtain a second sequence of channel signals. Then, based on a layout-affected sound effect compensation function, signal interference compensation is performed on the second sequence of channel signals according to the target vehicle's interior spatial layout model to obtain a third sequence of channel signals. Next, adaptive sound effect enhancement is performed on the third sequence of channel signals based on the target vehicle's in-vehicle and in-vehicle dynamic environment to obtain a fourth sequence of channel signals. Finally, the fourth sequence of channel signals is transmitted to multiple audio channels, and actual sound field feedback information is collected via an in-vehicle microphone array. Based on this actual sound field feedback, sound effect feedback optimization is performed on the multiple audio channels. This solves the technical problem in the prior art of lacking immersive in-vehicle sound effects due to dynamic changes in the vehicle's in-vehicle and in-vehicle environment. By using multi-channel collaborative processing, the immersiveness of in-vehicle sound effects is enhanced.

[0064] Embodiment 2 is based on the same inventive concept as the method for generating immersive in-vehicle sound effects by multi-channel collaborative processing in the aforementioned embodiment. Figure 2 As shown, the present application provides an immersive in-vehicle sound effect generation system for multi-channel collaborative processing, wherein the system includes:

[0065] An audio channel acquisition module 11 is used to set up multiple car audio systems in the target car to obtain multiple audio channels; a collaborative processing module 12 is used to perform multi-channel collaborative processing on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals; an immersive optimization module 13 is used to perform immersive optimization on the first sequence of channel signals according to the in-vehicle sound effect focusing factor of the target car to obtain a second sequence of channel signals; a sound effect compensation module 14 is used to perform signal interference compensation on the second sequence of channel signals according to the in-vehicle space layout model of the target car based on the layout impact sound effect compensation function to obtain a third sequence of channel signals; a sound effect enhancement module 15 is used to perform adaptive sound effect enhancement on the third sequence of channel signals according to the dynamic environment field inside and outside the target car to obtain a fourth sequence of channel signals; a feedback optimization module 16 is used to transmit the fourth sequence of channel signals to the multiple audio channels, collect actual sound field feedback information through the in-vehicle microphone array, and perform sound effect feedback optimization on the multiple audio channels according to the actual sound field feedback information.

[0066] Furthermore, the sound effect compensation module 14 is configured to perform the following method:

[0067] The layout-affected sound effect compensation function is:

[0068] Wherein, CSL2 represents any channel signal in the second sequence of channel signals, CSL3 represents the interference-compensated channel signal corresponding to CSL2, J represents the number of interference sources inside the target car, L space,j Characterizes the predicted interference value of the jth interference source on CSL2, j and J are both positive integers, 1≤j≤J, C j Characterization L space,j The corresponding compensation weight coefficient, β represents the error correction coefficient.

[0069] Furthermore, the sound effect compensation module 14 is configured to perform the following method:

[0070] According to the in-vehicle space layout model, sound interference sources are identified to determine multiple interference sources; any channel signal in the second sequence of channel signals is used as the first channel signal; interference prediction is performed on the first channel signal based on the multiple interference sources to determine a first interference prediction sequence; a proportion is calculated based on the first interference prediction sequence to obtain a first compensation weight sequence; based on the first interference prediction sequence and the first compensation weight sequence, signal interference compensation is performed on the first channel signal according to the layout impact sound compensation function to obtain the third sequence of channel signals.

[0071] Furthermore, the sound effect compensation module 14 is configured to perform the following method:

[0072] The plurality of interference sources are characterized according to the in-vehicle space layout model to obtain position features, structural features and material features of each interference source; P learners are supervised and trained according to the channel signal interference record set to obtain P channel signal interference prediction models, where P is a positive integer greater than 1; the first channel signal, the position features, structural features and material features of each interference source are input into the P channel signal interference prediction models to obtain a plurality of signal interference prediction sets corresponding to the plurality of interference sources; and central value calculations are performed on the plurality of signal interference prediction sets respectively to generate the first interference prediction sequence.

[0073] Furthermore, the sound enhancement module 15 is configured to perform the following method:

[0074] The dynamic environment fields inside and outside the vehicle include the dynamic environment field inside the vehicle and the dynamic environment field outside the vehicle; the sound effect loss of the third sequence of the channel signal is traced according to the dynamic environment field inside the vehicle to obtain a first sound effect loss tracing factor; the sound effect loss of the third sequence of the channel signal is traced according to the dynamic environment field outside the vehicle to obtain a second sound effect loss tracing factor; the first sound effect loss tracing factor and the second sound effect loss tracing factor are integrated to obtain a third sound effect loss tracing factor; the sound effect of the third sequence of the channel signal is enhanced according to the third sound effect loss tracing factor to generate the fourth sequence of the channel signal.

[0075] Furthermore, the sound enhancement module 15 is configured to perform the following method:

[0076] According to the dynamic environment field inside the vehicle, sound effect loss is predicted for each channel signal in the third sequence of the channel signal to obtain multiple signal sound effect loss coefficients; it is determined whether the multiple signal sound effect loss coefficients are greater than or equal to a sound effect loss threshold; if the multiple signal sound effect loss coefficients are greater than or equal to the sound effect loss threshold, the loss abnormal channel signal corresponding to the abnormal sound effect loss coefficient is determined; according to the loss abnormal channel signal, the dynamic environment field inside the vehicle is correlated and traced to obtain the first sound effect loss tracing factor.

[0077] Furthermore, the immersive optimization module 13 is configured to perform the following method:

[0078] When the in-vehicle sound effect focus factor is feature user priority, a first in-vehicle sound effect preference portrait is constructed based on the feature user's in-vehicle sound effect preference information; when the in-vehicle sound effect focus factor is vehicle-wide balance, data fusion is performed based on multiple in-vehicle user sound effect preference data to construct a second in-vehicle sound effect preference portrait; based on the first in-vehicle sound effect preference portrait or the second in-vehicle sound effect preference portrait, the first sequence of channel signals is adaptively immersive optimized to generate the second sequence of channel signals.

[0079] Furthermore, the collaborative processing module 12 is configured to execute the following method:

[0080] De-noising is performed on the sound source signal to obtain an enhanced sound source signal; response characteristics of the multiple audio channels are tested based on the enhanced sound source signal to establish a multi-channel response characteristic matrix; and multi-channel collaborative processing is performed on the enhanced sound source signal based on the multi-channel response characteristic matrix to generate the first sequence of channel signals.

[0081] Furthermore, the collaborative processing module 12 is configured to execute the following method:

[0082] A frequency response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel frequency response eigenvector; a phase response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel phase response eigenvector; a delay response confidence test is performed on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel delay response eigenvector; and the multi-channel response characteristic matrix is ​​constructed based on the multi-channel frequency response eigenvector, the multi-channel phase response eigenvector and the multi-channel delay response eigenvector.

[0083] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0084] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0085] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A method for generating immersive in-vehicle sound effects using multi-channel collaborative processing, characterized in that: The method comprises: Setting up multiple car stereos in the target car to obtain multiple audio channels; Performing multi-channel collaborative processing on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals; performing immersive optimization on the first sequence of channel signals according to the vehicle-mounted sound effect focusing factor of the target vehicle to obtain a second sequence of channel signals; Based on the layout-affected sound effect compensation function, performing signal interference compensation on the second sequence of the sound channel signal according to the interior space layout model of the target vehicle to obtain a third sequence of the sound channel signal; Adaptively enhance the sound effects of the third sequence of the sound channel signal according to the dynamic environment field inside and outside the target vehicle to obtain a fourth sequence of the sound channel signal; The fourth sequence of channel signals is transmitted to the multiple audio channels, actual sound field feedback information is collected through the in-vehicle microphone array, and sound effect feedback optimization is performed on the multiple audio channels based on the actual sound field feedback information.

2. The immersive vehicle sound effect generation method of multi-channel collaborative processing according to claim 1, characterized in that: The layout-affected sound effect compensation function is: ； Among them, CSL2 represents any channel signal in the second sequence of channel signals, CSL3 represents the interference-compensated channel signal corresponding to CSL2, J represents the number of interference sources in the target car, and L space,j Characterizes the predicted interference value of the jth interference source on CSL2, j and J are both positive integers, 1≤j≤J, C j Characterization L space,j The corresponding compensation weight coefficient, β represents the error correction coefficient.

3. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 1, wherein: Based on the layout-affected sound effect compensation function, signal interference compensation is performed on the second sequence of the channel signal according to the interior space layout model of the target vehicle to obtain a third sequence of the channel signal, including: Identify sound interference sources based on the interior space layout model and determine multiple interference sources; Taking any channel signal in the second sequence of channel signals as the first channel signal; Performing interference prediction on the first channel signal according to the multiple interference sources to determine a first interference prediction sequence; Calculate the proportions according to the first interference prediction sequence to obtain a first compensation weight sequence; Based on the first interference prediction sequence and the first compensation weight sequence, signal interference compensation is performed on the first channel signal according to the layout impact sound effect compensation function to obtain the third channel signal sequence.

4. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 3, wherein: Performing interference prediction on the first channel signal according to the multiple interference sources to determine a first interference prediction sequence includes: Performing feature recognition on the multiple interference sources according to the in-vehicle space layout model to obtain position features, structural features, and material features of each interference source; P learners are supervised and trained based on the vocal tract signal interference record set to obtain P vocal tract signal interference prediction models, where P is a positive integer greater than 1; Inputting the first channel signal, the position features of each interference source, the structural features of each interference source, and the material features of each interference source into the P channel signal interference prediction models to obtain multiple signal interference prediction sets corresponding to the multiple interference sources; Central value calculation is performed on each of the multiple signal interference prediction sets to generate the first interference prediction sequence.

5. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 1, wherein: Adaptively enhancing the sound effects of the third sequence of channel signals according to the dynamic environment field inside and outside the target vehicle to obtain a fourth sequence of channel signals includes: The dynamic environment field inside and outside the vehicle includes the dynamic environment field inside the vehicle and the dynamic environment field outside the vehicle; performing sound effect loss tracing on the third sequence of the sound channel signal according to the in-vehicle dynamic environment field to obtain a first sound effect loss tracing factor; performing sound effect loss tracing on the third sequence of the sound channel signal according to the external dynamic environment field to obtain a second sound effect loss tracing factor; Combining the first sound effect loss tracing factor and the second sound effect loss tracing factor to obtain a third sound effect loss tracing factor; The sound effect of the third sequence of the channel signal is enhanced according to the third sound effect loss tracing factor to generate the fourth sequence of the channel signal.

6. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 5, wherein: Tracing the source of sound loss of the third sequence of the sound channel signal according to the in-vehicle dynamic environment field to obtain a first factor of tracing the source of sound loss includes: Performing sound effect loss prediction on each channel signal in the third sequence of channel signals according to the in-vehicle dynamic environment field to obtain a plurality of signal sound effect loss coefficients; Determining whether the multiple signal sound effect loss coefficients are greater than or equal to a sound effect loss threshold; If the sound effect loss coefficients of the plurality of signals are greater than or equal to the sound effect loss threshold, determining a loss abnormal channel signal corresponding to the abnormal sound effect loss coefficient; The dynamic environment field in the vehicle is correlated and traced according to the abnormal loss sound channel signal to obtain the first factor of the sound effect loss tracing.

7. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 1, wherein: Immersively optimizing the first sequence of channel signals according to the vehicle-mounted sound effect focus factor of the target vehicle to obtain a second sequence of channel signals includes: When the in-vehicle sound effect focus factor is a characteristic user priority, constructing a first in-vehicle sound effect preference profile according to the characteristic user's in-vehicle sound effect preference information; When the in-vehicle sound effect focus factor is balanced for the entire vehicle, data fusion is performed based on the sound effect preference data of multiple in-vehicle users to construct a second in-vehicle sound effect preference profile; Adaptively perform immersion optimization on the first sequence of channel signals according to the first in-vehicle sound effect preference portrait or the second in-vehicle sound effect preference portrait to generate the second sequence of channel signals.

8. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 1, wherein: Performing multi-channel collaborative processing on the multiple audio channels according to the sound source signal to obtain a first sequence of channel signals includes: Performing denoising processing on the sound source signal to obtain an enhanced sound source signal; Performing response characteristic tests on the multiple audio channels according to the enhanced sound source signal to establish a multi-channel response characteristic matrix; Multi-channel collaborative processing is performed on the enhanced sound source signal according to the multi-channel response characteristic matrix to generate the first sequence of channel signals.

9. The method for generating immersive in-vehicle sound effects using multi-channel collaborative processing according to claim 8, wherein: Performing response characteristic tests on the multiple audio channels according to the enhanced sound source signal to establish a multi-channel response characteristic matrix includes: Performing a frequency response confidence test on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel frequency response feature vector; Performing a phase response confidence test on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel phase response feature vector; Performing a delay response confidence test on the multiple audio channels according to the enhanced sound source signal to obtain a multi-channel delay response feature vector; The multi-channel response characteristic matrix is ​​constructed according to the multi-channel frequency response eigenvector, the multi-channel phase response eigenvector, and the multi-channel delay response eigenvector.

10. An immersive in-vehicle sound effect generation system with multi-channel collaborative processing, characterized by: The method for generating immersive in-vehicle sound effects by implementing multi-channel collaborative processing according to any one of claims 1 to 9 comprises: An audio channel acquisition module is used to set up multiple car audio systems in the target car and obtain multiple audio channels; a collaborative processing module, configured to perform multi-channel collaborative processing on the plurality of audio channels according to the sound source signal to obtain a first sequence of channel signals; an immersive optimization module, configured to perform immersive optimization on the first sequence of channel signals according to the vehicle-mounted sound effect focus factor of the target vehicle to obtain a second sequence of channel signals; a sound effect compensation module, configured to perform signal interference compensation on the second sequence of sound channel signals based on a layout-affected sound effect compensation function and according to an interior space layout model of the target vehicle, to obtain a third sequence of sound channel signals; a sound effect enhancement module, configured to adaptively enhance the sound effect of the third sequence of the sound channel signal according to the dynamic environment field inside and outside the target vehicle to obtain a fourth sequence of the sound channel signal; A feedback optimization module is used to transmit the fourth sequence of channel signals to the multiple audio channels, collect actual sound field feedback information through the in-vehicle microphone array, and optimize the sound effect feedback of the multiple audio channels according to the actual sound field feedback information.

Images