Audio signal rendering method, renderer and vehicle-mounted audio system

Abstract

The invention provides an audio signal rendering method, a renderer and a vehicle-mounted audio system. The audio signal rendering method comprises the following steps: acquiring a sound bed signal to be rendered; based on a first system corresponding to the sound bed signal to be rendered and a target listening position, determining a conversion relation of the sound bed signal to be rendered, the conversion relation being used for describing a mapping relation between each loudspeaker sound channel signal of the first system and each loudspeaker sound channel signal of a second system; and converting the sound bed signal to be rendered based on the conversion relation to obtain a sound bed signal corresponding to each loudspeaker sound channel in the second system. Aiming at the rendering scene of the sound bed signal, the invention provides the rendering method based on audio sound channel conversion, which is suitable for sound bed signal rendering between an audio providing system and an audio playback system under various sound channel layouts, improves the accuracy of sound bed signal rendering, and improves the effect of sound bed rendering.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio signal rendering method, a renderer, and a vehicle-mounted audio system. Background Art

[0002] Atmos, also known as 3D sound, is a 3D immersive sound effect developed after 5.1-channel and 7.1-channel sound. Atmos uses the production concepts of bed audio, audio objects, and sound metadata, and restores the 3D sound field through immersive audio rendering technology, greatly improving the dynamic effect of sound. Atmos technology has also been widely used in vehicles.

[0003] For the sound bed signal, upmixing or downmixing technology is usually used to render the sound bed signal of the first system with fewer channels to the playback system with more channels, or to render the sound bed signal of the first system with more channels to the playback system with fewer channels. For the first system and the playback system with the same channel layout, the sound bed signal is directly rendered in a one-to-one correspondence manner.

[0004] Since the listener is not located at the center of the speaker array in the vehicle speaker playback environment, the rendering effect of the above-mentioned sound bed signal rendering method is poor and cannot meet the needs of immersive experience. Summary of the invention

[0005] The present application provides an audio signal rendering method, a renderer and a vehicle-mounted audio system. The conversion of the sound bed signal is realized by mapping the speaker channel signals between different systems in the conversion relationship, and the gain of each speaker channel when rendering the sound bed signal in the playback environment is obtained, thereby realizing the combination of upmixing technology and downmixing technology, and improving the effect of sound bed signal rendering.

[0006] In a first aspect, the present application provides an audio signal rendering method, comprising:

[0007] Get the sound bed signal to be rendered;

[0008] Determining, based on the first system corresponding to the sound bed signal to be rendered and the target listening position, a conversion relationship of the sound bed signal to be rendered, wherein the conversion relationship is used to describe a mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system;

[0009] The sound bed signal to be rendered is converted based on the conversion relationship to obtain a sound bed signal corresponding to each speaker channel in the second system.

[0010] In some embodiments, determining the conversion relationship of the sound bed signal to be rendered based on the first system corresponding to the sound bed signal to be rendered and the target listening position includes:

[0011] Acquire a speaker layout of the first system and a speaker layout of the second system;

[0012] The conversion relationship is determined based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position.

[0013] In some embodiments, determining the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position includes:

[0014] Based on a preset correspondence, a conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position is determined, wherein the preset correspondence is used to describe the correspondence between the speaker layout of the first system, the speaker layout of the second system and the listening position and the conversion relationship.

[0015] In some embodiments, the speaker layout includes a channel layout and a position layout, and determining the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position includes:

[0016] Determine, based on the position layout of the first system speaker layout, the position layout of the second system speaker layout, and the target listening position, the second speaker channels corresponding to the first speaker channels, wherein the first speaker channels are speaker channels in the first system, and the second speaker channels are speaker channels in the second system;

[0017] For each of the first speaker channels, based on a channel spatial position of the first speaker channel and a channel spatial position of each of the second speaker channels corresponding to the first speaker channel, determining a gain of each of the second speaker channels corresponding to the first speaker channel with respect to the first speaker channel;

[0018] For each second speaker channel, determining a gain vector of the second speaker channel based on the gain of the second speaker channel with respect to each corresponding first speaker channel;

[0019] The conversion relationship is determined based on the gain vectors of the second speaker channels.

[0020] In some embodiments, when the number of the second speaker channels is greater than 3, determining the second speaker channels corresponding to the first speaker channels based on the position layout of the first system speaker layout, the position layout of the second system speaker layout, and the target listening position includes:

[0021] Determining a channel spatial position of each of the first speaker channels and a channel spatial position of each of the second speaker channels from the position layout of the first system speaker layout and the position layout of the second system speaker layout respectively;

[0022] For each of the first speaker channels, based on the channel spatial position of the first speaker channel and the target listening position, a connecting line corresponding to the first speaker channel is obtained; when the second speaker channel does not include a sky channel and the first speaker channel includes a sky channel, the connecting line corresponding to the sky channel in the first speaker channel is a connecting line obtained by taking the position of the projection speaker channel corresponding to the sky channel in the first speaker channel and the target listening position as endpoints, and the projection speaker channel is obtained by projecting the corresponding sky channel onto the plane where the horizontal channel in the first speaker channel is located;

[0023] When the second speaker channel includes a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space surface intersected by a line corresponding to the first speaker channel;

[0024] When the second speaker channel does not include a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space connecting line intersecting with a connecting line corresponding to the first speaker channel;

[0025] The channel space plane is a plane composed of three adjacent second speaker channels including a sky channel; and the channel space line is a line connecting two adjacent second speaker channels.

[0026] In some embodiments, the method further comprises:

[0027] Determining whether the preset corresponding relationship includes a conversion relationship corresponding to the speaker layout of the second system;

[0028] If yes, determining a conversion relationship under the speaker layout of the first system, the speaker layout of the second system and the target listening position based on the preset corresponding relationship;

[0029] If not, the step of determining the second speaker channels corresponding to the first speaker channels based on the position layout of the first system speaker layout, the position layout of the second system speaker layout and the target listening position is performed, and after determining the conversion relationship based on the gain vector of each second speaker channel, the correspondence between the speaker layout of the second system, the speaker layout of the first system and the target listening position and the determined conversion relationship is added to the preset correspondence.

[0030] In some embodiments, obtaining a sound bed signal to be rendered includes:

[0031] Obtaining the sound bed signal to be rendered and the sound object to be rendered obtained by decoding the audio signal to be rendered sent by the first system by the decoder;

[0032] The method further comprises:

[0033] The sound object to be rendered is rendered to obtain a sound object signal corresponding to each speaker in the second system.

[0034] In some embodiments, the second system is an audio playback system of a vehicle, and the method further comprises:

[0035] When a plurality of users are seated in the vehicle, the target listening position is determined based on the seat positions of the respective users, the position priorities of the seat positions of the users, and the priorities of the respective users.

[0036] In a second aspect, the present application further provides an audio signal rendering device, comprising:

[0037] A sound bed signal acquisition module, used to acquire the sound bed signal to be rendered;

[0038] a conversion relationship determination module, configured to determine a conversion relationship of the sound bed signal to be rendered based on the first system corresponding to the sound bed signal to be rendered and a target listening position, wherein the conversion relationship is used to describe a mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system;

[0039] The sound bed signal rendering module is used to convert the sound bed signal to be rendered based on the conversion relationship to obtain the sound bed signal corresponding to each speaker channel in the second system.

[0040] In a third aspect, the present application further provides a renderer, which is arranged in a second system and is used to execute the audio signal rendering method provided in the first aspect of the present application.

[0041] In a fourth aspect, the present application also provides a vehicle-mounted audio system, which is arranged in a vehicle and includes multiple speakers and the renderer provided in the third aspect of the present application.

[0042] In a fifth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the processor implements the audio signal method provided in the first aspect of the present application.

[0043] In a sixth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the audio signal method provided in the first aspect of the present application.

[0044] The audio signal rendering method, renderer and vehicle audio system provided in the present application are aimed at application scenarios of sound bed signal rendering, such as rendering scenarios of sound bed signals in vehicles. For the sound bed signal to be rendered provided by the first system, based on the distribution of speakers in the first system and the position of the target listening position in the second system, the mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system is determined, and the conversion relationship of the sound bed signal to be rendered is obtained, and then the sound bed signal to be rendered is converted based on the conversion relationship to obtain the sound bed signal corresponding to each speaker channel in the second system, so as to achieve the rendering of the sound bed signal. By determining the conversion relationship, the sound bed signal of the first system under various channel layouts is converted into the sound bed signal of the second system under various channel layouts, which improves the accuracy of the sound bed signal conversion, and then improves the effect of the sound bed signal rendering. In combination with the panoramic sound speakers, panoramic sound rendering can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0046] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0047] Figure 2 A flowchart of an audio signal rendering method provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a flow chart of another audio signal rendering method provided in an embodiment of the present application;

[0049] Figure 4 For this application Figure 1 A schematic diagram of a second speaker channel partitioning result provided by the illustrated embodiment;

[0050] Figure 5 A schematic diagram of the positional relationship between a first speaker channel, a second speaker channel, and a target listening position provided in one embodiment of the present application;

[0051] Figure 6 For this application Figure 3 A schematic diagram of the process of step S303 in the illustrated embodiment;

[0052] Figure 7 A schematic diagram of determining a corresponding second speaker channel by connecting a channel space according to an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of a vehicle audio system provided in an embodiment of the present application.

[0054] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0056] In the field of sound bed rendering technology, down-mixing or up-mixing technology is usually used for rendering. Up-mixing is used to transform the audio signal with fewer channels into an audio signal with more channels when the number of input channels is less than the number of output channels; down-mixing is used to transform the audio signal with more channels into an audio signal with fewer channels when the number of input channels is greater than the number of output channels.

[0057] In some embodiments, a channel is also called a channel.

[0058] For example, the upmixing technology may be used to process the sound bed signal from stereo 2.0 channels to three-dimensional stereo 3.0 channels, and the downmixing technology may be used to process the sound bed signal from 7.1 channels to 5.1 channels.

[0059] In traditional upmixing or downmixing technology, the correspondence between input speakers and output speakers is default or manually specified. The input speakers are the speakers of the system that provides the sound bed signal (referred to as the first system), and the output speakers are the speakers of the system that renders the sound bed signal provided by the first system (referred to as the second system). This results in low accuracy when mixing the sound bed signals and limits the scenarios that can be processed. For example, it cannot be applied to the scenario of rendering the sound bed signals of the first system and the second system with the same channel layout.

[0060] In the case where the first system and the second system have the same channel layout, such as 7.1.3 channel layout, the relevant technology directly sends the signal of the same channel in the first system to the corresponding channel of the second system.

[0061] The first system may be any system that can provide audio, such as an audio recording system, and the second system may be any system that plays back audio with a speaker, such as a home theater audio system, a car audio system, etc.

[0062] In response to the above problems, the present application provides an audio signal rendering method, which determines the correspondence between the channel signals of each speaker in the second system and the channel signals of each speaker in the first system based on the system corresponding to the sound bed signal to be rendered, that is, the first system, and the position of the listening position, that is, obtains the conversion relationship of the sound bed signal to be rendered, and converts the sound bed signal to be rendered through the conversion relationship to achieve the rendering of the sound bed signal. By determining the conversion relationship, a rendering of the sound bed signal under various channel layouts is provided, which has a wide range of applications. At the same time, since the conversion relationship is determined based on the first system and the listening position, the accuracy of the sound bed signal conversion is improved, and the effect of the sound bed signal rendering is improved.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application, Figure 1 For example, the first system is an audio production system and the second system is a car audio system. Figure 1 In the embodiment, the channel layout of the first system and the second system is 7.1.4 channels. The 7.1.4 channel car audio system includes 7 horizontal speakers (or surround speakers), 1 subwoofer and 4 sky speakers (or top speakers). Figure 1 Solid dots are used to represent horizontal speakers, dotted circles are used to represent subwoofers, and solid circles are used to represent sky speakers.

[0065] The layout of the speakers in the audio production system is usually quite regular, and the listening position is the center of the speaker array, that is, point O1. In a vehicle, the layout of the speakers in the in-vehicle audio production system is affected by the shape of the vehicle. At the same time, the listening position in the vehicle is usually the head of the user sitting in the vehicle seat, and the listening position deviates from the center of the speaker array in the in-vehicle audio system, that is, point O2.

[0066] For sound bed rendering of an audio production system and a car audio system with the same channel layout, the sound bed signal of the audio production system is usually directly sent to the same-named speaker of the car audio system to achieve the rendering of the sound bed signal. However, due to the different position layout and channel layout of the speakers of the audio production system and the car audio system, the sound bed rendering effect is poor.

[0067] The audio signal rendering method provided in the present application can not only realize the rendering of sound bed signals between systems with different channel layouts, but also realize the rendering of sound bed signals between different systems with the same channel layout, through the conversion relationship determined based on the listening position and the channel layout of the first system. It also takes into account the specific position of the listening position, thereby improving the three-dimensional effect of the rendering.

[0068] Figure 2 A flowchart of an audio signal rendering method provided in an embodiment of the present application is provided. The method is applied to a second system, which is a system that needs to render an audio signal, and can be a car audio system, a cinema audio system, or other audio systems. The method can be executed by a renderer of the second system, or by a server or other remote device.

[0069] like Figure 2 As shown, the audio signal rendering method includes the following steps:

[0070] Step S201, obtaining a sound bed signal to be rendered.

[0071] The sound bed signal to be rendered is a sound bed signal that needs to be rendered in the second system. It can be a sound bed signal of an audio signal corresponding to music, video, etc., or it can be a sound bed signal of an audio signal output by an application program such as in-car karaoke, games, etc.

[0072] In some embodiments, the to-be-rendered sound bed signal is specifically a sound bed signal output by a decoder of the second system.

[0073] In some embodiments, the input audio signal may be decomposed by the decoder into a sound object (AudioObject), a bed audio signal (Bed Audio) and sound metadata, wherein the bed audio signal is the bed audio signal to be rendered.

[0074] Specifically, an audio signal to be rendered is obtained, and based on a decoder, the audio signal to be rendered is decomposed into a sound object, a sound bed signal, and sound metadata, and subsequent steps are performed using the sound bed signal as the sound bed signal to be rendered.

[0075] Step S202, based on the first system corresponding to the sound bed signal to be rendered and the target listening position, determine the conversion relationship of the sound bed signal to be rendered, wherein the conversion relationship is used to describe the mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system.

[0076] In some embodiments, the conversion relationship is specifically used to describe the gain of each speaker channel of the second system when rendering each speaker channel signal of the first system.

[0077] The first system corresponding to the to-be-rendered sound bed signal may be stored in metadata corresponding to the to-be-rendered sound bed signal.

[0078] The target listening position is the listening position in the second system, and rendering is performed for the target listening position to optimize the auditory experience of the target listening position. The conversion relationship is used to describe the corresponding relationship of the gain on which the sound bed signal of each speaker channel of the first system is converted into the sound bed signal of each speaker channel of the second system.

[0079] The target listening position may be a designated location in the second system, such as a target listening position uploaded or selected by a user.

[0080] The target listening position may also be the position of the head of the identified user. When there are multiple users, a target may be selected from the multiple users, and the head of the target user may be used as the target listening position. The target listening position may also be calculated based on the positions of the heads of multiple users, such as the target listening position being the average of the positions of the heads of multiple users. The target listening position may also be determined based on the user level and the positions of the heads of multiple users, specifically by determining the weight based on the user level, and taking the weighted average of the positions of the heads of multiple users as the target listening position.

[0081] The target listening position may be determined based on a selection instruction input by the user.

[0082] In some embodiments, the selection instructions may include the coordinates of the target listening position.

[0083] In other embodiments, the selection instruction may include a selected seat, such as a main driver's seat, a front passenger seat, a left rear seat, etc., such as a seat in a theater, and the coordinates of the target listening position are determined based on the selected seat.

[0084] The user may input a selection instruction through the interactive device, and after receiving the selection instruction, the renderer of the second system determines the coordinates of the target listening position based on the selection instruction.

[0085] In some embodiments, the selection instruction can be sent to a remote device, such as a server, a user terminal, etc., through the vehicle's networking device, and the remote device obtains the coordinates of the target listening position based on the selection instruction and executes subsequent conversion relationship determination steps.

[0086] Taking the second system as an in-vehicle audio system as an example, the target listening position is a position in the vehicle, which can be a designated position, such as the front passenger seat, the middle seat in the back row, etc., or a position determined based on the situation of the user in the vehicle. The coordinates of the target listening position can be the coordinates in a three-dimensional coordinate system with the left front wheel of the vehicle as the origin, or the coordinates in a spherical coordinate system with the center of the speaker array in the vehicle as the origin.

[0087] When there are multiple users in the vehicle, the coordinates of the center of the head of any one of the users can be used as the coordinates of the target listening position, such as the user who frequently rides the vehicle, the user with the highest priority, etc.

[0088] Optionally, the second system is an audio playback system of a vehicle (or an in-vehicle audio system), and the method further includes:

[0089] When a plurality of users are seated in the vehicle, the target listening position is determined based on the seat positions of the respective users, the position priorities of the seat positions of the users, and the priorities of the respective users.

[0090] The weight of each user's seating position may be determined based on the priority of the user's seating position and the priority of the user, and the weighted average of the user's seating positions may be determined as the target listening position.

[0091] The weight of the back row listening position can be set higher than that of the front row listening position, and the weight of the listening position at the front passenger seat can be set higher than that at the main driver's seat. The higher the priority of the user, the higher the weight of the position where the user sits.

[0092] By automatically determining the target listening position, the automation level of the sound bed rendering is improved. At the same time, combining the position level and the user level, the target listening position is determined from multiple positions, which improves the accuracy of the listening position determination and improves the overall auditory effect of multiple users.

[0093] The conversion relationship can be expressed by a matrix, in which the elements in the i-th row and j-th column are used to describe the gain of the i-th second speaker channel under the j-th first speaker channel. The first speaker channel is the speaker of the first system, and the second speaker channel is the speaker of the second system.

[0094] The matrix T is a sparse matrix. Taking a simple 5.1.0-channel recording speaker system (a first system) configured to a 5.1.0-channel car audio system (a second system), with the main driver's listening position as the target listening position as an example, the conversion matrix T in this scenario is:

[0095]

[0096] Among them, α LL With α CL are respective mapping coefficients (or gains) for mapping the L channel and the C channel in the first speaker channel (the speaker channel of the first system) to the L channel in the second speaker channel (the speaker channel of the second system); α RR With α CR are the respective mapping coefficients for mapping the R channel and the C channel in the first speaker channel to the R channel in the playback speaker channel; α LC , αRC With α CC are the respective mapping coefficients of the L channel, the R channel and the C channel in the first speaker channel to the C channel in the second speaker channel; α LrsLrs is the mapping coefficient of the Lrs channel in the first speaker channel to the Lrs channel in the second speaker channel; α RrsRrs The mapping coefficient of the Rrs channel in the first speaker channel to the Rrs channel of the second speaker channel. Among them, the L channel is the left channel, the R channel is the right channel, the C channel is the center channel, the Lrs channel is the left rear surround channel, and the Rrs channel is the right rear surround channel.

[0097] The values ​​of the elements in the matrix corresponding to the conversion relationship can be obtained through a variety of ways, including but not limited to: based on the experience of the tuner, calculated through acoustic theory, and inferred through a pre-trained model, which can be a deep learning model, a machine learning model, a statistical model or other models.

[0098] In some embodiments, the conversion relationships in various scenarios may be pre-arranged by a tuner, and the scenarios may be determined by the target listening position, the first system, and the second system.

[0099] Specifically, the above conversion relationship may be determined based on the speaker layout in the first system and the target listening position. The speaker layout may include a channel layout and / or a position layout.

[0100] When determining the conversion relationship, for the convenience of calculation, the listening position of the first system needs to be overlapped with the target listening position of the second system in the virtual space, so as to determine the positional relationship between the speakers or speaker channels in the first system and the second system.

[0101] Optionally, determining a conversion relationship of the sound bed signal to be rendered based on the first system corresponding to the sound bed signal to be rendered and a target listening position includes:

[0102] Acquire the speaker layout of the first system and the speaker layout of the second system; and determine the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position.

[0103] The speaker layout may include a channel layout, such as 7.1.4 channels, 7.3.4 channels, etc., and may also include a position layout.

[0104] The channel layout in the speaker layout of the first system may be the same as or different from the channel layout in the speaker layout of the second system.

[0105] In some embodiments, the number of speaker channels in the second system is greater than three.

[0106] Specifically, based on the speaker layout of the first system, the speaker layout of the second system and the target listening position, the second speaker channels corresponding to the first speaker channels can be determined, and then based on the distance from the second speaker channels corresponding to the first speaker channels to the target listening position and the distance from the first speaker channels to the corresponding second speaker channels, the gains of the second speaker channels corresponding to the first speaker channels when rendering the first speaker channel signals can be determined. For each second speaker channel, the gains of the second speaker channels when rendering the corresponding first speaker channel signals are accumulated to obtain the gains corresponding to the second speaker channels, and the second speaker channels are traversed to obtain the above conversion relationship.

[0107] Optionally, determining the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position includes:

[0108] Based on a preset correspondence, a conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position is determined, wherein the preset correspondence is used to describe the correspondence between the speaker layout of the first system, the speaker layout of the second system and the listening position and the conversion relationship.

[0109] When the sound bed signal to be rendered is detected, based on the speaker layout and target listening position of the first system corresponding to the sound bed signal to be rendered, the conversion relationship under the speaker layout and target listening position of the first system is found from the preset conversion relationship corresponding to the second system.

[0110] In some embodiments, the preset correspondence relationship may be a conversion relationship list corresponding to the second system stored in advance, and the conversion relationship under the speaker layout of the first system and the target listening position may be searched from the conversion relationship list corresponding to the second system. The conversion relationship list corresponding to the second system includes conversion relationships corresponding to the second system under different speaker layouts of the first system and different listening positions.

[0111] Taking the second system as an in-vehicle audio system as an example, for various in-vehicle audio systems, a conversion relationship list corresponding to the in-vehicle audio system can be pre-configured, that is, the conversion relationship of the sound bed signal to be rendered of the second system under different listening positions and different first systems is pre-determined to obtain the conversion relationship list corresponding to the second system.

[0112] For vehicles, the target listening positions are usually relatively fixed. Common listening positions for a 5-seater vehicle include the main driver's seat, the co-driver's seat, the left rear listening position, the right rear listening position, and the center rear listening position. For each fixed listening position in each vehicle model, the conversion relationship of the listening position under various speaker layouts of the first system is respectively configured.

[0113] Step S203: convert the sound bed signal to be rendered based on the conversion relationship to obtain a sound bed signal corresponding to each speaker channel in the second system.

[0114] After the conversion relationship is determined, the sound bed signals of the speaker channels of each first system in the sound bed signals to be rendered are converted and superimposed based on the conversion relationship to obtain the sound bed signals corresponding to each speaker channel of the second system.

[0115] The sound bed signal corresponding to each speaker channel in the second system can be obtained by multiplying the sound bed signal to be rendered by the conversion relationship.

[0116] Taking the conversion relationship as matrix T as an example, the sound bed signal corresponding to each speaker channel in the speaker array corresponding to the second system can be expressed as:

[0117]

[0118] Where n represents the time series number; A signal vector representing a sound bed signal corresponding to each speaker channel in the speaker array corresponding to the second system; A signal vector representing a sound bed signal corresponding to each speaker channel of the speaker array corresponding to the first system.

[0119] Through the sound bed rendering based on the above conversion relationship, a local upmixing and local downmixing rendering solution can be implemented.

[0120] In the technical solution provided in this embodiment, for application scenarios of sound bed signal rendering, such as the rendering scenario of sound bed signal in a vehicle, for the sound bed signal to be rendered provided by the first system, based on the distribution of the speakers in the first system and the position of the target listening position in the second system, the gain of the speakers in the second system under each speaker in the first system is determined to obtain the conversion relationship of the sound bed signal to be rendered, and then the sound bed signal to be rendered is converted based on the conversion relationship to obtain the sound bed signal corresponding to each speaker channel in the second system, thereby achieving the rendering of the sound bed signal. By converting the conversion relationship determined based on the listening position and the first system, the sound bed signal of the first system under various channel layouts is converted into the sound bed signal of the second system under various channel layouts, thereby improving the accuracy of the sound bed signal conversion, and further improving the effect of the sound bed signal rendering. In combination with the panoramic sound speakers, panoramic sound bed rendering can be achieved.

[0121] Figure 3 A flowchart of another audio signal rendering method provided in an embodiment of the present application. Figure 2 Based on the embodiment shown, step S201 and step S202 are further refined, and a sound object rendering step is added, such as Figure 3 As shown, the audio signal rendering method provided in this embodiment may include the following steps:

[0122] Step S301: Obtain the sound bed signal to be rendered and the sound object to be rendered obtained by decoding the audio signal to be rendered sent by the first system by the decoder.

[0123] After obtaining the audio signal to be rendered through the speaker array recording of the first system, the audio signal to be rendered is sent to the decoder corresponding to the second system, and the decoder decodes the audio signal to be rendered to obtain the sound bed signal to be rendered and the sound object to be rendered.

[0124] Step S302: Render the sound object to be rendered to obtain a sound object signal corresponding to each speaker channel in the second system.

[0125] For the sound object to be rendered, any rendering algorithm can be used to render it, and the sound object signal corresponding to each speaker channel in the second system can be obtained, thereby combining the sound bed signal corresponding to each speaker channel to achieve the rendering of the audio signal to be rendered.

[0126] The rendering algorithm of the sound object to be rendered may be a panning rendering algorithm, a wave field synthesis rendering algorithm (Wave Field Synthesis, WFS), a higher order panoramic sound rendering algorithm (Higher Oder Ambisonics, HOA) or other algorithms, which is not limited in this application.

[0127] The panning rendering algorithms include basic panning algorithms and improved panning algorithms. The improved panning algorithms include Vector Based Amplitude Panning (VBAP) and Distance Based Amplitude Panning (DBAP).

[0128] The second speaker is a speaker in the second system, and the first speaker is a speaker in the first system.

[0129] In some embodiments, based on the coordinates of the target listening position and the coordinates of each second speaker, a synthetic sound intensity function of each second speaker at the target listening position can be determined, and based on the synthetic sound intensity, the gain of each second speaker channel when rendering the sound object to be rendered can be determined to achieve the rendering of the sound object to be rendered. The synthetic sound intensity function is a function that describes the synthetic sound intensity of the second speaker with respect to the gain of each second speaker channel when rendering the sound object to be rendered.

[0130] Specifically, the virtual sound intensity of the virtual sound source corresponding to the sound object to be rendered at the target listening position can be determined; the gain of each second speaker channel when rendering the sound object to be rendered is solved by reconstructing the virtual sound intensity function through the above-mentioned synthetic sound intensity function, and the sound object signal corresponding to each speaker channel in the second system is obtained.

[0131] Based on the Green function, the sound pressure function of the virtual sound source corresponding to the sound object to be rendered can be described, and combined with the acoustic Euler equation, the virtual sound intensity of the virtual sound source at the target listening position can be obtained.

[0132] The synthetic sound intensity function includes a first part and a second part, the first part is a function of the active sound intensity synthesized by the sound pressure and vibration velocity generated by the speaker corresponding to the same second speaker channel in the active sound intensity of the synthetic sound intensity, and the second part is a function of the active sound intensity generated by the coupling between speakers of different second speaker channels in the active sound intensity of the synthetic sound intensity; the active sound intensity of the virtual sound intensity is set equal to the above-mentioned first part, with the goal of minimizing the second part, and the gain of each second speaker channel when rendering the sound object to be rendered is solved.

[0133] Specifically, the sound pressure of the second speaker at the target listening position can be determined based on the coordinates of the target listening position, as well as the coordinates, frequency and orientation of the second speaker; based on the sound pressure of each second speaker at the target listening position, the composite sound intensity of each second speaker at the target listening position can be obtained.

[0134] The Green's Function can be used to describe the sound pressure of the sound source corresponding to each second speaker. The sound source corresponding to the second speaker can be regarded as a point sound source or a surface sound source. Whether the sound source corresponding to the second speaker is a point sound source or a surface sound source can be determined based on the distance of the second speaker relative to the target listening position.

[0135] Exemplarily, when the distance between the second speaker and the target listening position is less than or equal to a preset distance, the sound source corresponding to the second speaker is determined to be a surface sound source; otherwise, the sound source corresponding to the second speaker is determined to be a point sound source.

[0136] The preset distance may be obtained based on the size of the second speaker, such as a preset multiple of the size of the second speaker, where the preset multiple may be 8 times, 10 times, 15 times or other multiples. The size of the speaker may be a speaker diaphragm radius.

[0137] For a second loudspeaker with a point sound source, the spherical wave of the point sound source corresponding to the second loudspeaker is described by a Green's function to obtain a sound pressure function of the second loudspeaker.

[0138] For the second speaker with surface sound source, several points can be taken on the second speaker grille as point sound sources, and the sound pressures of the several point sound sources corresponding to the second speaker can be described by Green's function to obtain spherical waves of the sound pressures of the several point sound sources. The spherical waves of the several point sound sources can be superimposed to obtain the sound pressure function of the second speaker with surface sound source.

[0139] Step S303: determining each second speaker channel corresponding to each first speaker channel based on the position layout of the first system speaker layout, the position layout of the second system speaker layout and the target listening position.

[0140] The first speaker channel is a speaker channel in the first system, and the second speaker channel is a speaker channel in the second system.

[0141] The number of speaker channels in the second system may be 2, 3, or even greater than 3, which is not limited in this embodiment.

[0142] Based on the position or coordinates of each speaker in the position layout of the first system speaker layout, the position or coordinates of each speaker in the position layout of the second system speaker layout, and the coordinates of the target listening position, one or more second speaker channels corresponding to the first speaker channel are determined.

[0143] When the position layout of the first system speaker layout and the position layout of the second system speaker layout use different coordinate systems, the method also includes: performing coordinate conversion on the position layout of the first system speaker layout so that the position layout of the first system speaker layout and the position layout of the second system speaker layout use the same coordinate system to describe the coordinates of the speakers.

[0144] Specifically, the speaker channels in the second system can be partitioned based on the coordinates of the second speaker channel to obtain multiple speaker areas, where one speaker area corresponds to at least one second speaker channel; and the second speaker channel corresponding to the speaker area that intersects with the line connecting the first speaker channel and the target listening position is determined to be the second speaker channel corresponding to the first speaker channel.

[0145] If there are frequency-dividing speakers in the second speaker channel, the multiple frequency-dividing speakers may be regarded as a virtual speaker, and the virtual speaker may be located at the center of the speaker area or other positions.

[0146] The speaker area can be a triangle, square or other geometric shapes.

[0147] The division of the speaker areas may only consider the outer surfaces of the three-dimensional shapes corresponding to the speakers of the second system, and each outer surface corresponds to a speaker area.

[0148] When dividing the loudspeaker zones, only horizontal loudspeakers and overhead loudspeakers (if any) can be considered.

[0149] Taking a vehicle as an example, the speaker zones may be located within a space within the vehicle.

[0150] For the second system including overhead speakers, the speaker area may be a plane consisting of three adjacent speaker channels including at least one overhead channel.

[0151] For example, Figure 4 For this application Figure 1 The schematic diagram of the second speaker channel partitioning result provided by the illustrated embodiment is as follows: Figure 4 As shown, the speakers of the second system with 7.1.4 channels are divided into multiple areas (triangular or quadrilateral areas), and the area division is performed based on the positions of the horizontal channels and the sky channels in the second system to obtain multiple speaker areas. Figure 4 In the figure, the horizontal channels include L (Left, left front), C (Center, center), R (Right, right front), Rss (Right SideSurround, right surround), Rrs (Right Rear Surround, right rear surround), Lrs (Left Rear Surround, left rear surround) and Lss (Left Side Surround, left surround) seven channels, the sky channels include Ltf (Left TopFront, left front top), Rtf (Right Top Front, right front top), Rtr (Right Top Rear, right rear top) and Ltr (Left Top Rear, left rear top) four channels, LEF represents the bass channel.

[0152] The multiple speaker areas include: a surface composed of all horizontal channels (solid dots), a surface composed of all sky channels (hollow circles), a surface composed of a sky channel and two adjacent horizontal channels, a surface composed of two adjacent sky channels and horizontal channels adjacent to the two adjacent sky channels, such as a surface composed of the sky channel of the Ltf channel and the Rtf channel and the center channel C, and a surface composed of the Ltr channel, the Rtr channel, the Lrs channel and the Rrs channel.

[0153] The target listening position is inside the three-dimensional shape corresponding to the spatial position of the speaker channel of the second system. The second speaker channel corresponding to the speaker area through which the line connecting the first speaker channel and the target listening position passes is the second speaker channel corresponding to the first speaker channel.

[0154] Figure 5 A schematic diagram of the positional relationship between a first speaker channel, a second speaker channel, and a target listening position provided in one embodiment of the present application is shown in FIG. Figure 5 As shown, the channel layout of the first system and the second system are both 7.1.4 channels. Figure 5 In the figure, solid lines are used to represent the speaker channels of the first system and dotted lines are used to represent the speaker channels of the second system. Taking the L channel of the first system as an example, the target listening position is the main driving position of the vehicle, and the speaker area passed by the line 51 connecting the L channel of the first system and the target listening position is the area corresponding to the surface composed of the L channel, C channel and Ltf channel in the second system. Then, the second speaker channel corresponding to the L channel of the first system is the L channel, C channel and Ltf channel in the second system.

[0155] Step S304: for each of the first speaker channels, based on the channel spatial position of the first speaker channel and the channel spatial position of each of the second speaker channels corresponding to the first speaker channel, determine the gain of each of the second speaker channels corresponding to the first speaker channel with respect to the first speaker channel.

[0156] When the speaker channel (the first speaker channel or the second speaker channel) includes a cross-frequency speaker, the channel spatial position of the speaker channel may be an average value of the positions of the cross-frequency speakers in the speaker channel or the position of any cross-frequency speaker in the speaker channel; when the speaker channel does not include a cross-frequency speaker, the channel spatial position of the speaker channel is the position of the speaker in the speaker channel.

[0157] After determining the second speaker channels corresponding to each first speaker channel, in order to locally upmix the sound bed signal of the first speaker channel to the corresponding multiple second speaker channels, it is necessary to determine the gain of each second speaker channel with respect to the corresponding first speaker channel based on the channel spatial position of the first speaker channel and the channel spatial position of each second speaker channel corresponding to the first speaker channel, that is, to determine the gain when the second speaker is used to render the corresponding first speaker channel signal.

[0158] Specifically, based on the distance between the first speaker channel and the target listening position, and the distance between each second speaker channel corresponding to the first speaker channel and the target listening position, the gain of each second speaker channel with respect to the corresponding first speaker channel can be determined, so that the sum of the energies decomposed into each corresponding second speaker channel by the determined gain is equal to the energy of the first speaker channel.

[0159] Assume that the energy of the first loudspeaker channel c1 at the target listening position is: in, is the position vector of the first loudspeaker channel c1, is the position vector of the target listening position. The gain of each second speaker channel corresponding to the first speaker channel can be calculated by the following formula:

[0160]

[0161]

[0162] Wherein, C1 is the number of second loudspeaker channels corresponding to the first loudspeaker channel c1; are the coordinates of the lth frequency division loudspeaker of the second loudspeaker channel c2 in the second system; is the number of crossover loudspeakers in the second loudspeaker channel c2 in the second system; is the energy ratio of the lth frequency division loudspeaker of the second loudspeaker channel c2 in the second system, and the energy ratio is a known parameter, is the directivity intensity of the lth frequency division loudspeaker of the second loudspeaker channel c2; is the angle between the normal direction of the lth frequency division loudspeaker of the second loudspeaker channel c2 and the line connecting it to the target listening position. When the second loudspeaker channel does not include a frequency division loudspeaker, the corresponding parameters in the formula (such as ) takes the value of 1; is the gain of the second speaker channel c2 relative to the first speaker channel c1; T is the normalized parameter to be solved.

[0163] Substitution Solving the above equation, we get:

[0164]

[0165] Then get for:

[0166]

[0167] The above formula is explained by taking a complex scenario of a frequency-dividing speaker as an example. For a non-frequency-dividing speaker, the gain of the second speaker channel of the non-frequency-dividing speaker with respect to the first speaker channel can be obtained by simplifying the above formula.

[0168] Step S305 : for each second speaker channel, determine a gain vector of the second speaker channel based on the gain of the second speaker channel with respect to each corresponding first speaker channel.

[0169] After obtaining the gain of each second speaker channel with respect to the corresponding first speaker channel, for each second speaker channel, the gain of the second speaker channel with respect to the corresponding one or more first speaker channels is combined to obtain a gain vector of the second speaker channel.

[0170] Step S306: determining the conversion relationship based on the gain vector of each of the second speaker channels.

[0171] The gain vectors of the second speaker channels are combined to obtain the above conversion relationship, such as a conversion matrix.

[0172] Step S307: convert the sound bed signal to be rendered based on the conversion relationship to obtain a sound bed signal corresponding to each speaker channel in the second system.

[0173] For each second speaker channel, based on the gain vector of the second speaker channel in the conversion relationship and the sound bed signals of each first speaker channel corresponding to the second speaker channel, the sound bed signal corresponding to the second speaker channel is obtained through local downmixing technology, that is, the product of the gain of the second speaker channel with respect to the corresponding first speaker channels and the sound bed signal of the first speaker channel is superimposed to obtain the sound bed signal corresponding to the second speaker channel.

[0174] For the second speaker channel c2, its corresponding sound bed signal for:

[0175]

[0176] Wherein, n is the time serial number; C2 is the number of the first speaker channels corresponding to the second speaker channel c2; is the gain of the second loudspeaker channel c2 with respect to the first loudspeaker channel c1; It is the sound bed signal to be rendered for the first speaker channel c1.

[0177] In this embodiment, the conversion relationship between the first speaker channel and the second speaker channel is determined by combining the position layout of the two system speakers and the position of the target listening position, and the conversion relationship is formulated in real time for specific scenarios, thereby improving the accuracy of the conversion relationship determination, realizing the sound bed rendering technology combining local upmixing and local downmixing, improving the accuracy of sound bed signal rendering, and combining with the rendering of sound objects to realize panoramic sound rendering of audio signals, thereby improving the rendering effect.

[0178] In some embodiments, in order to save the amount of calculation for the sound bed to render the same scene continuously and improve the rendering efficiency, the conversion relationships obtained in each scene can be sorted out to obtain a preset corresponding relationship. Then, when the sound bed is rendered next time, it can be determined whether there is a conversion relationship in the scene in the preset corresponding relationship. If so, there is no need to perform the step of determining the conversion relationship based on the speaker layout in the above embodiment, and the conversion relationship in the scene can be directly obtained from the preset corresponding relationship to perform sound bed signal conversion.

[0179] Optionally, the method further includes:

[0180] Determine whether the preset correspondence includes a conversion relationship corresponding to the speaker layout of the second system; if so, determine the conversion relationship under the speaker layout of the first system, the speaker layout of the second system and the target listening position based on the preset correspondence; if not, perform the step of determining each second speaker channel corresponding to each first speaker channel based on the position layout of the speaker layout of the first system, the position layout of the speaker layout of the second system and the target listening position, and after determining the conversion relationship based on the gain vector of each second speaker channel, add the correspondence between the speaker layout of the second system, the speaker layout of the first system and the target listening position and the determined conversion relationship to the preset correspondence.

[0181] The preset corresponding relationship can be searched based on the speaker layout of the second system. If the conversion relationship corresponding to the speaker layout of the second system is found, the speaker layout of the first system and the conversion relationship under the target listening position are filtered out from the conversion relationship corresponding to the speaker layout of the second system, so as to convert the speaker sound bed signal of the first system based on the filtered conversion relationship to obtain the sound bed signal corresponding to each second speaker channel.

[0182] If the conversion relationship corresponding to the speaker layout of the second system is not found from the preset corresponding relationship, execute Figure 3In step S303, a conversion relationship under the scenario (the speaker layout of the first system, the speaker layout of the second system and the target listening position) is determined.

[0183] Optionally, when the number of the second speaker channels is greater than 3, Figure 6 For this application Figure 3 The flowchart of step S303 in the embodiment shown is as follows: Figure 6 As shown, step S303 may specifically include the following steps:

[0184] Step S601: determining a channel spatial position of each first speaker channel and a channel spatial position of each second speaker channel from the position layout of the first system speaker layout and the position layout of the second system speaker layout respectively.

[0185] The spatial position of the speaker's sound channel can be expressed using spherical coordinates or a position vector.

[0186] Step S602: for each of the first speaker channels, based on the channel spatial position of the first speaker channel and the target listening position, obtain a connection line corresponding to the first speaker channel.

[0187] In general, the channel spatial position of the first speaker channel and the target listening position can be directly used as endpoints to obtain the connection line corresponding to the first speaker channel. In the special case where the first speaker channel is the sky channel and the second speaker channel does not include the sky channel, it is necessary to project the sky channel onto the horizontal plane (the plane where the horizontal channel is located), and use the projection of the sky channel on the horizontal plane and the target listening position as endpoints to obtain the connection line corresponding to the sky channel.

[0188] When the second speaker channel does not include a sky channel and the first speaker channel includes a sky channel, the connecting line corresponding to the sky channel in the first speaker channel is a connecting line obtained with the position of the projection speaker channel corresponding to the sky channel in the first speaker channel and the target listening position as endpoints, and the projection speaker channel is obtained by projecting the corresponding sky channel onto the plane where the horizontal channel in the first speaker channel is located.

[0189] When the second speaker channel does not include the sky channel and the first speaker channel includes the sky channel, it is necessary to project the sky channel in the first speaker channel onto the horizontal plane where the horizontal channel in the first speaker channel is located, and obtain the connecting line of the sky channel in the first speaker channel with the projection of the sky channel in the first speaker channel on the horizontal plane (i.e., the position of the projected speaker channel) and the target listening position as two endpoints.

[0190] The sky channel projection method is adopted to avoid the problem that the line connecting the sky channel and the target listening position does not intersect with the channel space line connecting the second speaker channel, thereby failing to determine the second speaker channel corresponding to the sky speaker in the first speaker channel.

[0191] Step S603: when the second speaker channel includes a sky channel, determining a plurality of second speaker channels associated with the first speaker channel based on a channel space plane intersected by a line corresponding to the first speaker channel.

[0192] The channel space plane is a plane composed of three adjacent second speaker channels including the sky channel.

[0193] For the second system including the sky channel, such as 7.1.4 channels, 7.3.4 channels, 9.1.4 channels, etc., the second speaker channel corresponding to the first speaker channel may be determined by a channel space surface through which a connecting line corresponding to the first speaker channel passes or intersects.

[0194] Exemplarily, the channel space plane may be a plane composed of an L channel speaker, a C channel speaker, and an Ltf channel speaker, or may be a plane composed of an R channel speaker, a C channel speaker, and an Rtf channel speaker.

[0195] Step S604: when the second speaker channel does not include a sky channel, determining a plurality of second speaker channels associated with the first speaker channel based on a channel space line intersecting with a line corresponding to the first speaker channel.

[0196] The channel space connection line is a connection line between two adjacent second speaker channels.

[0197] When the speakers of the second system do not include a sky speaker or the second speaker channel does not include a sky channel, the aforementioned channel space surface needs to be degenerated into a channel space line, and the two second speaker channels corresponding to the channel space lines that intersect with the lines corresponding to the first speaker channels are determined to be the second speaker channels corresponding to the first speaker channels.

[0198] For example, Figure 7 A schematic diagram of determining a corresponding second speaker channel by connecting a channel space provided by an embodiment of the present application, in Figure 7 A first loudspeaker channel is represented by a solid line and a second loudspeaker channel is represented by a dotted line, Figure 7As shown, the speaker channels of the second system are 5.1 channels, including 5 horizontal channels, namely channels 711 to 715, and the speaker channels of the first system are 7.1.4, including 4 sky channels and 7 horizontal channels (channels 721 to 727). The sky channels of the first system are first projected onto the horizontal plane to obtain projections 731 to 734. Projection 731 is the projection of the Ltf channel of the first system. The line 735 connecting the projection 731 and the target listening position O7 intersects with the channel space line 741. The channel space line 741 is the line connecting channel 711 and channel 712. It is determined that the second speaker channels corresponding to the Ltf channels of the first system are channels 711 and channel 712. The second speaker channels corresponding to other first speaker channels can be obtained in a similar manner.

[0199] By determining the second speaker channel corresponding to the first speaker channel in the above manner, the channel layout and position layout of the speakers in the corresponding scene are fully considered, the accuracy of speaker channel correspondence is improved, and thus the accuracy of gain calculation is improved.

[0200] An audio signal rendering device provided in an embodiment of the present application includes:

[0201] A sound bed signal acquisition module is used to acquire the sound bed signal to be rendered; a conversion relationship determination module is used to determine the conversion relationship of the sound bed signal to be rendered based on the first system corresponding to the sound bed signal to be rendered and the target listening position, and the conversion relationship is used for the mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system; a sound bed signal rendering module is used to convert the sound bed signal to be rendered based on the conversion relationship to obtain the sound bed signal corresponding to each speaker channel in the second system.

[0202] In some embodiments, the conversion relationship determination module includes:

[0203] A layout acquisition unit is used to acquire the speaker layout of the first system and the speaker layout of the second system; a conversion relationship determination unit is used to determine the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position.

[0204] In some embodiments, the conversion relationship determination unit is specifically configured to:

[0205] Based on a preset correspondence, a conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position is determined, wherein the preset correspondence is used to describe the correspondence between the speaker layout of the first system, the speaker layout of the second system and the listening position and the conversion relationship.

[0206] In some embodiments, the speaker layout includes a channel layout and a position layout, and the conversion relationship determination unit includes:

[0207] A speaker corresponding subunit is used to determine each second speaker channel corresponding to each first speaker channel based on the position layout of the speaker layout of the first system, the position layout of the speaker layout of the second system and the target listening position, wherein the first speaker channel is a speaker in the first system and the second speaker channel is a speaker in the second system; a gain determination subunit is used to determine, for each first speaker channel, the gain of each second speaker channel corresponding to the first speaker channel with respect to the first speaker channel based on the channel spatial position of the first speaker channel and the channel spatial position of each second speaker channel corresponding to the first speaker channel; a conversion relationship determination subunit is used to determine, for each second speaker channel, the gain vector of the second speaker channel based on the gain of the second speaker channel with respect to each corresponding first speaker channel, and determine the conversion relationship based on the gain vector of each second speaker channel; when the speaker is a cross-frequency speaker, the channel spatial position of the speaker is the average value of the positions of each cross-frequency speaker in the same channel; when the speaker is not a cross-frequency speaker, the channel spatial position of the speaker is the position of the speaker.

[0208] In some embodiments, when the number of the second speaker channels is greater than 3, the speaker corresponding subunit is specifically configured to:

[0209] Determining a channel spatial position of each of the first speaker channels and a channel spatial position of each of the second speaker channels from the position layout of the first system speaker layout and the position layout of the second system speaker layout respectively;

[0210] For each of the first speaker channels, based on the channel spatial position of the first speaker channel and the target listening position, a connecting line corresponding to the first speaker channel is obtained; when the second speaker channel does not include a sky channel and the first speaker channel includes a sky channel, the connecting line corresponding to the sky channel in the first speaker channel is a connecting line obtained by taking the position of the projection speaker channel corresponding to the sky channel in the first speaker channel and the target listening position as endpoints, and the projection speaker channel is obtained by projecting the corresponding sky channel onto the plane where the horizontal channel in the first speaker channel is located;

[0211] When the second speaker channel includes a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space surface intersected by a connecting line corresponding to the first speaker channel;

[0212] When the second speaker channel does not include a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space connecting line intersecting with a connecting line corresponding to the first speaker channel;

[0213] The channel space plane is a plane composed of three adjacent second speaker channels including a sky channel; and the channel space line is a line connecting two adjacent second speaker channels.

[0214] In some embodiments, the apparatus further comprises:

[0215] The preset correspondence determination unit is used to determine whether the preset correspondence includes a conversion relationship corresponding to the speaker layout of the second system.

[0216] Accordingly, the conversion relationship determination unit is specifically used for:

[0217] If the preset corresponding relationship includes a conversion relationship corresponding to the speaker layout of the second system, the conversion relationship under the speaker layout of the first system, the speaker layout of the second system and the target listening position is determined based on the preset corresponding relationship; if the preset corresponding relationship does not include a conversion relationship corresponding to the speaker layout of the second system, the conversion relationship is determined by the following steps:

[0218] Based on the position layout of the first system speaker layout, the position layout of the second system speaker layout and the target listening position, determine each second speaker channel corresponding to each first speaker channel, wherein the first speaker channel is a speaker in the first system, and the second speaker channel is a speaker in the second system; for each first speaker channel, based on the position of the first speaker channel and the position of each second speaker channel corresponding to the first speaker channel, determine the gain of each second speaker channel corresponding to the first speaker channel with respect to the first speaker channel; for each second speaker channel, based on the gain of the second speaker channel with respect to each corresponding first speaker channel, determine the gain vector of the second speaker channel; based on the gain vector of each second speaker channel, determine the conversion relationship.

[0219] In some embodiments, the acoustic bed signal acquisition module is specifically used to:

[0220] The sound bed signal to be rendered and the sound object to be rendered are obtained after the decoder decodes the audio signal to be rendered sent by the first system.

[0221] Accordingly, the device further comprises:

[0222] The object rendering module is used to render the sound object to be rendered to obtain the sound object signal corresponding to each speaker in the second system.

[0223] In some embodiments, the second system is an audio playback system of a vehicle, and the apparatus further comprises:

[0224] The target listening position determination module is used to determine the target listening position based on the seating position of each user, the position priority of the user seating position and the priority of each user when there are multiple users in the vehicle.

[0225] The above device embodiment is an embodiment corresponding to the above method embodiment, and has the same technical effect as the method embodiment. The detailed description of the device embodiment can refer to the detailed description of the above method embodiment, and will not be repeated here.

[0226] An embodiment of the present application also provides a renderer, which is arranged in a vehicle, and is used to execute the audio signal rendering method provided by any embodiment of the present application to achieve panoramic sound rendering of the sound bed signal.

[0227] Figure 8 The structure diagram of a vehicle audio system provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the vehicle audio system includes: multiple speakers and a renderer.

[0228] The renderer is used to execute the audio signal rendering method provided in any embodiment of the present application, obtain the gain of each of the multiple second speaker channels when rendering the corresponding audio signal, and realize the panoramic sound rendering of the audio signal.

[0229] In some embodiments, the in-vehicle audio system further includes a plurality of sensors.

[0230] In some embodiments, the in-vehicle audio system includes an ADAS (Advanced Driving Assistance System).

[0231] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the processor implements a method provided in any embodiment of the present application.

[0232] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the method provided in any embodiment of the present application.

[0233] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0234] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for rendering an audio signal, characterized in that: include: Get the sound bed signal to be rendered; Determining, based on the first system corresponding to the sound bed signal to be rendered and the target listening position, a conversion relationship of the sound bed signal to be rendered, wherein the conversion relationship is used to describe a mapping relationship between the channel signals of each speaker of the first system and the channel signals of each speaker of the second system; The sound bed signal to be rendered is converted based on the conversion relationship to obtain a sound bed signal corresponding to each speaker channel in the second system.

2. The method according to claim 1, characterized in that Determining a conversion relationship of the sound bed signal to be rendered based on the first system corresponding to the sound bed signal to be rendered and the target listening position includes: Acquire a speaker layout of the first system and a speaker layout of the second system; The conversion relationship is determined based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position.

3. The method according to claim 2, characterized in that Determining the conversion relationship based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position includes: Based on a preset correspondence, a conversion relationship based on the speaker layout of the first system, the speaker layout of the second system and the target listening position is determined, wherein the preset correspondence is used to describe the correspondence between the speaker layout of the first system, the speaker layout of the second system and the listening position and the conversion relationship.

4. The method according to claim 2, characterized in that: The speaker layout includes a channel layout and a position layout. Based on the speaker layout of the first system, the speaker layout of the second system, and the target listening position, determining the conversion relationship includes: Determine, based on the position layout of the first system speaker layout, the position layout of the second system speaker layout, and the target listening position, the second speaker channels corresponding to the first speaker channels, wherein the first speaker channels are speaker channels in the first system, and the second speaker channels are speaker channels in the second system; For each of the first speaker channels, based on a channel spatial position of the first speaker channel and a channel spatial position of each of the second speaker channels corresponding to the first speaker channel, determining a gain of each of the second speaker channels corresponding to the first speaker channel with respect to the first speaker channel; For each second speaker channel, determining a gain vector of the second speaker channel based on the gain of the second speaker channel with respect to each corresponding first speaker channel; The conversion relationship is determined based on the gain vectors of the second speaker channels.

5. The method according to claim 4, characterized in that When the number of the second speaker channels is greater than 3, determining the second speaker channels corresponding to the first speaker channels based on the position layout of the first system speaker layout, the position layout of the second system speaker layout and the target listening position includes: Determining a channel spatial position of each of the first speaker channels and a channel spatial position of each of the second speaker channels from the position layout of the first system speaker layout and the position layout of the second system speaker layout respectively; For each of the first speaker channels, based on the channel spatial position of the first speaker channel and the target listening position, a connecting line corresponding to the first speaker channel is obtained; when the second speaker channel does not include a sky channel and the first speaker channel includes a sky channel, the connecting line corresponding to the sky channel in the first speaker channel is a connecting line obtained by taking the position of the projection speaker channel corresponding to the sky channel in the first speaker channel and the target listening position as endpoints, and the projection speaker channel is obtained by projecting the corresponding sky channel onto the plane where the horizontal channel in the first speaker channel is located; When the second speaker channel includes a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space surface intersected by a connecting line corresponding to the first speaker channel; When the second speaker channel does not include a sky channel, determining a plurality of second speaker channels corresponding to the first speaker channel based on a channel space connecting line intersecting with a connecting line corresponding to the first speaker channel; The channel space plane is a plane composed of three adjacent second speaker channels including a sky channel; and the channel space line is a line connecting two adjacent second speaker channels.

6. The method according to claim 4, characterized in that The method further comprises: Determining whether the preset corresponding relationship includes a conversion relationship corresponding to the speaker layout of the second system; If yes, determining a conversion relationship under the speaker layout of the first system, the speaker layout of the second system and the target listening position based on the preset corresponding relationship; If not, the step of determining the second speaker channels corresponding to the first speaker channels based on the position layout of the first system speaker layout, the position layout of the second system speaker layout and the target listening position is performed, and after determining the conversion relationship based on the gain vector of each second speaker channel, the correspondence between the speaker layout of the second system, the speaker layout of the first system and the target listening position and the determined conversion relationship is added to the preset correspondence.

7. The method according to any one of claims 1 to 6, characterized in that: Get the sound bed signal to be rendered, including: Obtaining the sound bed signal to be rendered and the sound object to be rendered obtained by decoding the audio signal to be rendered sent by the first system by the decoder; The method further comprises: The sound object to be rendered is rendered to obtain a sound object signal corresponding to each speaker in the second system.

8. The method according to any one of claims 1 to 6, characterized in that: The second system is an audio playback system of the vehicle, and the method further includes: When a plurality of users are seated in the vehicle, the target listening position is determined based on the seat positions of the users, the position priorities of the seat positions of the users, and the priorities of the users.

9. A renderer, characterized in that: The renderer is arranged in the second system, and the renderer is used to execute the method according to any one of claims 1 to 8 to render the audio signal.

10. A vehicle audio system, arranged in a vehicle, characterized in that: include: A plurality of speakers and the renderer as claimed in claim 9.

Images