Vehicle-mounted audio rendering method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle-mounted audio rendering method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a vehicle-mounted pickup signal input by a vehicle-mounted user and a pickup signal rendering parameter, and determining a rendering parameter calculation result corresponding to a vehicle-mounted loudspeaker sound channel system when the vehicle-mounted pickup signal is rendered based on the pickup signal rendering parameter; rendering the vehicle-mounted pickup signal based on the rendering parameter calculation result to obtain a pickup rendering signal; and acquiring a background rendering signal corresponding to the vehicle-mounted pickup signal, performing signal mixing processing on the pickup rendering signal and the background rendering signal to obtain a target audio signal, and sending the target audio signal to each vehicle-mounted loudspeaker. According to the method, the overall rendering effect of the audio signal is improved.

Description

Technical Field

[0001] The present application relates to the field of audio rendering technology, and in particular to a vehicle-mounted audio rendering method, device, electronic device and storage medium. Background Art

[0002] With the development of the economy, especially the development of the automobile industry, the number of cars in society is increasing. With the improvement of people's living standards, it has become very common to use cars as a means of transportation. Consumers have higher and higher requirements for vehicle configuration when buying cars.

[0003] For example, traditional in-car playback devices that only have music playback functions can no longer meet the active participation of car users in the music experience. Therefore, the in-car karaoke function was born.

[0004] However, when multiple irregular speakers are arranged in a vehicle, it is difficult for existing audio signal rendering algorithms to accurately render the sound pickup signals input by users during karaoke and the audio signals of background music, resulting in poor overall rendering effect of the target audio in the car. Summary of the invention

[0005] The present application provides a vehicle-mounted audio rendering method, device, electronic device and storage medium, which are used to solve the problem of poor rendering effect of target audio in the vehicle in the audio rendering method in the prior art. The method separately renders the picked-up signal of the vehicle user and the background signal, and determines the rendering gain of each speaker based on the picked-up signal rendering parameter of the picked-up signal before rendering the picked-up signal, and then renders the picked-up signal based on the rendering gain, and mixes the rendered picked-up rendering signal and the rendered audio rendering signal to achieve accurate rendering of the audio signal, thereby optimizing the overall rendering effect in the vehicle.

[0006] In a first aspect, the present application provides a vehicle audio rendering method, comprising: obtaining a vehicle sound pickup signal and a sound pickup signal rendering parameter input by a vehicle user, and determining a rendering parameter calculation result corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal based on the sound pickup signal rendering parameter;

[0007] Rendering the vehicle-mounted sound pickup signal based on the rendering parameter calculation result to obtain a sound pickup rendering signal;

[0008] A background rendering signal corresponding to the vehicle-mounted sound pickup signal is obtained, the sound pickup rendering signal and the background rendering signal are mixed to obtain a target audio signal, and the target audio signal is sent to each of the vehicle-mounted speakers.

[0009] Optionally, the sound pickup signal rendering parameters include the listening position parameters of the vehicle user and the sound source parameters of the virtual sound source corresponding to the vehicle sound pickup signal;

[0010] The sound source parameters include sound source position parameters and sound source shape parameters; the sound source position parameters include sound source distance and sound source angle; the sound source shape parameters include uniform shape parameters or non-uniform shape parameters;

[0011] The uniform shape parameters include the uniform size of the sound source; the non-uniform shape parameters include the height, width and depth of the sound source.

[0012] Optionally, the virtual sound source corresponding to the vehicle-mounted sound pickup signal includes multiple auxiliary virtual sound sources;

[0013] Accordingly, after obtaining the rendering parameters of the sound pickup signal input by the vehicle user, the method further includes:

[0014] Determine the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal based on the sound source shape parameter and a preset shape threshold; the sound source type includes a point sound source and a volume sound source;

[0015] If it is determined that the virtual sound source is a point sound source, the virtual sound source positions of the auxiliary virtual sound sources are the same, and the sound source position parameters are determined as the virtual sound source position parameters of the virtual sound source;

[0016] If the virtual sound source is determined to be a volume sound source, the virtual sound source positions of the auxiliary virtual sound sources are different, and virtual sound source position parameters corresponding to the auxiliary virtual sound sources are determined based on the sound source position parameters and the shape parameters.

[0017] Optionally, the rendering parameter calculation result includes a channel rendering gain, if the virtual sound source is a point sound source;

[0018] Correspondingly, the calculation result of the rendering parameter corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal based on the sound pickup signal rendering parameter includes:

[0019] Determine, based on the listening position parameter and the virtual sound source position parameter, a corresponding rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source;

[0020] A gain calculation method corresponding to the rendering mode is obtained, and based on the gain calculation method, a channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined.

[0021] Optionally, the rendering mode includes near-field rendering or far-field rendering;

[0022] Correspondingly, the determining, based on the listening position parameter and the virtual sound source position parameter, of a corresponding rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source includes:

[0023] Acquiring a sound field threshold rendered by the vehicle-mounted speaker signal, and comparing the virtual sound source position parameter with the sound field threshold;

[0024] If the virtual sound source position parameter is within the sound field threshold range, determining that the rendering mode is near-field rendering;

[0025] If the virtual sound source position parameter is outside the sound field threshold range, the rendering mode is determined to be far-field rendering.

[0026] Optionally, if the rendering mode is near-field rendering;

[0027] Accordingly, determining the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method includes:

[0028] Determine a first distance between each of the vehicle-mounted speakers and a virtual sound source position corresponding to the vehicle-mounted sound pickup signal based on the virtual sound source position parameter and the speaker position parameter of each of the vehicle-mounted speakers, and determine a normalized parameter for rendering the vehicle-mounted sound pickup signal based on each of the first distances;

[0029] A channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined based on the normalized parameter and each of the first distances.

[0030] Optionally, if the rendering mode is far-field rendering;

[0031] Accordingly, determining the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method includes:

[0032] Acquire an attenuation parameter of the vehicle-mounted sound pickup signal rendering, and determine a normalization parameter of the vehicle-mounted sound pickup signal rendering based on the attenuation parameter and each of the first distances;

[0033] A channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined based on the normalized parameter and each of the first distances.

[0034] Optionally, the rendering parameter calculation result includes a channel rendering gain, if it is determined that the virtual sound source is a volume sound source;

[0035] Correspondingly, the calculation result of the rendering parameter corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal based on the sound pickup signal rendering parameter includes:

[0036] For any auxiliary virtual sound source, determining a rendering mode corresponding to rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the listening position parameter and the virtual sound source position parameter of the current auxiliary virtual sound source;

[0037] Acquire a gain calculation method corresponding to the rendering mode, and determine, based on the gain calculation method, a sound source rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the current auxiliary virtual sound source;

[0038] Based on the sound source rendering gain of each of the auxiliary virtual sound sources, a channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source is determined.

[0039] Optionally, the acquiring a background rendering signal corresponding to the vehicle-mounted sound pickup signal, and performing signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain a target audio signal includes:

[0040] Acquire a background audio file corresponding to the vehicle-mounted sound pickup signal, and perform decoding preprocessing on the background audio file to obtain a preprocessed background audio signal;

[0041] Based on a preset audio rendering algorithm, the background audio signal is rendered to obtain a rendered background rendering signal;

[0042] Based on a preset signal mixing algorithm, the corresponding channel signals of the sound pickup rendering signal and the background rendering signal are superimposed and mixed to obtain a target audio signal.

[0043] In a second aspect, the present application provides a vehicle-mounted audio rendering device, comprising:

[0044] A rendering parameter calculation result determination module, used to obtain a vehicle-mounted sound pickup signal and a sound pickup signal rendering parameter input by a vehicle-mounted user, and determine a rendering parameter calculation result corresponding to a vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal based on the sound pickup signal rendering parameter;

[0045] A sound pickup rendering signal obtaining module, used for rendering the vehicle-mounted sound pickup signal based on the rendering parameter calculation result to obtain a sound pickup rendering signal;

[0046] The target audio signal acquisition module is used to obtain the background rendering signal corresponding to the vehicle-mounted sound pickup signal, perform signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain the target audio signal, and send the target audio signal to each of the vehicle-mounted speakers.

[0047] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0048] The memory stores computer-executable instructions;

[0049] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0050] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0051] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method of the first aspect when the computer program is executed by a processor.

[0052] The technical solution provided by the present application determines in advance the calculation result of the rendering parameters corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal based on the rendering parameters of the sound pickup signal input by the user before rendering the vehicle sound pickup signal input by the user, and renders the vehicle sound pickup signal based on the rendering parameter calculation result to obtain the sound pickup rendering signal, so as to achieve accurate rendering of the sound pickup signal; at the same time, renders the background audio signal corresponding to the vehicle sound pickup signal to obtain the corresponding background rendering signal, and mixes the rendered sound pickup rendering signal and the background rendering signal and transmits them to the speaker for playback, thereby achieving accurate rendering of the audio signal in the scenario where the speakers in the vehicle are irregularly arranged, thereby optimizing the overall rendering effect in the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] Figure 1 An application scenario diagram of the vehicle audio rendering method provided in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of a flow chart of a vehicle audio rendering method provided in an embodiment of the present application;

[0056] Figure 3A schematic diagram of an interface for inputting sound pickup signal rendering parameters provided in an embodiment of the present application;

[0057] Figure 4 A flowchart of another in-vehicle audio rendering method provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of the layout of the vehicle-mounted speakers provided in an embodiment of the present application;

[0059] Figure 6 A schematic cross-sectional view of a sound field sphere constructed in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a vehicle-mounted audio rendering device provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0062] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present application.

[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0065] In actual applications, with the improvement of people's living standards, consumers are no longer satisfied with safety configurations when choosing vehicles, and their requirements for entertainment configurations are also getting higher and higher. For example, in-vehicle playback devices that only have music playback functions can no longer meet users' current needs for music experience. Therefore, in order to meet the needs of consumers, some vehicles have added karaoke functions to the in-vehicle playback devices, allowing car users to immersively participate in the music playback process, thereby improving the user experience of music playback participation.

[0066] Since the music playback environment in the car is different from the playback scenes in traditional cinemas, concert halls, theaters and other playback environments where speakers are regularly arranged, the irregular layout of multiple car speakers in the car makes it difficult to accurately render the background audio signals and the car pickup signals input by the car user when singing in the car using the existing traditional rendering methods, resulting in poor overall rendering effect of the rendered target audio.

[0067] The in-vehicle audio rendering method provided by the present application aims to solve the above technical problems of the prior art. Specifically, by separately rendering the in-vehicle sound pickup signal and the background audio signal of the in-vehicle user, and determining the rendering parameter calculation result corresponding to the in-vehicle speaker channel system when rendering the in-vehicle sound pickup signal based on the sound pickup signal rendering parameter of the in-vehicle sound pickup signal before rendering the in-vehicle sound pickup signal, and then rendering the in-vehicle sound pickup signal based on the rendering parameter calculation result, and mixing the rendered sound pickup rendering signal and the background rendering signal obtained by rendering the background audio signal in the background audio file, so as to achieve accurate rendering of the audio signal, thereby optimizing the overall rendering effect in the car.

[0068] Figure 1 This is an application scenario diagram of the in-vehicle audio rendering method provided in the embodiment of the present application. Figure 1 The application scenarios to which the embodiments of the present application are applicable are described. Figure 1 , specifically including a parameter acquisition device, a signal acquisition device, an audio file storage device and an audio processing device. Specifically, the audio file storage module is used to store the background audio file when the user sings in the car in the form of karaoke. When the user in the car determines the song he is singing, the background audio file corresponding to the song is called from the audio file storage module, and the background audio file is transmitted to the audio processing device for audio signal rendering processing; further, the signal acquisition device may include a wireless and / or wired microphone to collect the vehicle-mounted sound pickup signal input by the user in the car when singing, and transmit the collected vehicle-mounted sound pickup signal to the audio processing device for subsequent audio rendering processing. It should be noted that in order to achieve accurate rendering of the vehicle-mounted sound pickup signal in the car, the technical solution of the present application also needs to collect the sound pickup signal rendering parameters input by the user before rendering the vehicle-mounted sound pickup signal, and transmit the sound pickup signal rendering parameters to the audio processing device, so as to achieve an accurately rendered target audio signal in the layout scenario where multiple vehicle-mounted speakers are irregularly arranged in the car.

[0069] Optionally, the audio processing device may include a background audio rendering module, a sound pickup signal rendering module, and a signal mixing module. Specifically, the background audio rendering module receives the background audio signal transmitted by the audio file storage device, and renders the background audio signal to obtain a corresponding background rendering signal; at the same time, the sound pickup signal rendering module receives the vehicle sound pickup signal and the sound pickup signal rendering parameter transmitted by the signal acquisition device, and processes the sound pickup signal rendering parameter to obtain the rendering parameter calculation result corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal, and renders the vehicle sound pickup signal according to the rendering parameter calculation result to obtain the corresponding sound pickup rendering signal; further, the signal mixing module mixes the vehicle sound pickup signal and the background rendering signal to obtain a target audio signal, and then sends the target audio signal to each speaker in the vehicle for audio playback.

[0070] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0071] Figure 2 The flowchart of a vehicle audio rendering method provided in an embodiment of the present application is shown in FIG. The method can be executed by a vehicle audio rendering device, which can be a server or an electronic device. The following description takes an electronic device as an example. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware, such as Figure 2 As shown, the method includes the following steps.

[0072] S210: Acquire an in-vehicle sound pickup signal and a rendering parameter of the sound pickup signal input by an in-vehicle user, and determine, based on the rendering parameter of the sound pickup signal, a calculation result of a rendering parameter corresponding to a channel system of an in-vehicle speaker when rendering the in-vehicle sound pickup signal.

[0073] In the embodiment of the present application, the in-vehicle user can be understood as a user who sings in the form of karaoke in the car for entertainment. The in-vehicle sound pickup signal can be understood as the sound pickup signal input by the signal acquisition device when the user sings in the car. It should be further explained that in order to enable the in-vehicle sound pickup signal input by the user when singing to be accurately rendered in the playback scenario where multiple irregular speakers are arranged in the car, the present application can also pre-collect the sound pickup signal rendering parameters corresponding to the in-vehicle sound pickup signal before rendering the in-vehicle sound pickup signal, and then determine the rendering parameter calculation results corresponding to the rendering of the in-vehicle sound pickup signal to each in-vehicle speaker based on the sound pickup signal rendering parameters, so as to accurately render the in-vehicle sound pickup signal based on the rendering parameter calculation results, thereby optimizing the overall rendering effect in the car, and then satisfying the user's immersive experience effect when the rendered target audio signal is played.

[0074] Among them, the sound pickup signal rendering parameters include the listening position parameters of the vehicle user and the sound source parameters of the virtual sound source corresponding to the vehicle-mounted sound pickup signal; the sound source parameters include sound source position parameters and sound source shape parameters; the sound source position parameters include sound source distance and sound source angle; the sound source shape parameters include uniform shape parameters or non-uniform shape parameters; the uniform shape parameters include the uniform size of the sound source; the non-uniform shape parameters include the sound source height, width and sound source depth.

[0075] Specifically, the listening position can be understood as the best listening position for the audio played by the speakers in the car. Correspondingly, the listening position parameters can be understood as the position coordinate data of the listening position in the preset vehicle coordinate system. In practical applications, the listening position can be pre-set to preset positions such as the main driver's seat, the co-driver's seat, and the rear seat according to the user's choice. Of course, it can also be set to other positions in the car, and this application does not make specific restrictions on this.

[0076] The sound source parameters can be understood as the three-dimensional spatial parameters of the virtual sound source that emits the sound perceived by the user in the car when the vehicle-mounted sound pickup signal is rendered and played by the speaker. Among them, the sound source parameters can specifically include the sound source position parameters. Further, the sound source position parameters also specifically include the sound source distance and the sound source angle. Specifically, the sound source distance is the distance parameter between the virtual sound source and the listening position; the sound source angle is the pitch angle parameter between the virtual sound source and the listening position and the horizontal angle parameter between the virtual sound source and the listening position.

[0077] In practical applications, the pitch angle reflects the height change of the virtual sound source relative to the listening position; the horizontal angle reflects the horizontal direction change of the virtual sound source relative to the listening position; and the above distance, pitch angle and horizontal angle can be combined to specifically determine the specific sound source position of the virtual sound source corresponding to the vehicle-mounted sound pickup signal after rendering.

[0078] In order to further enhance the listening experience of users in the car, the sound source parameters may also specifically include sound source shape parameters. In practical applications, the sound source shape can be characterized as the shape of the virtual sound source that the user in the car perceives when the vehicle-mounted sound pickup signal is rendered and played by the speaker. Among them, the sound source shape parameters include uniform shape parameters or non-uniform shape parameters. Specifically, the uniform shape parameters may include the shape size of the sound source; the non-uniform shape parameters may include parameters such as the sound source height, width, and sound source depth.

[0079] In this application, the rendering parameter calculation result can be understood as the rendering adjustment parameter used by the renderer to render the vehicle sound pickup signal using the speaker channel. Specifically, the rendering adjustment parameter can be a rendering adjustment parameter in other forms such as channel rendering gain, speaker channel rendering matrix, speaker channel filter signal, etc. This application does not limit the specific expression of the rendering adjustment parameter.

[0080] In practical applications, when the rendering parameter calculation result is a channel rendering gain, in order to achieve accurate playback of audio signals, high-frequency speakers and low-frequency speakers are usually arranged separately in the car. That is, the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle-mounted sound pickup signal is determined by calculating the sound pickup signal rendering parameters, and the vehicle-mounted sound pickup signal is processed by the channel rendering gain rendering to obtain the sound pickup rendering signal. After obtaining the target audio signal, the target audio signal will be divided into different frequencies, and the divided audio signal will be transmitted to the divided frequency speakers for playback.

[0081] On the basis of the above, the vehicle-mounted sound pickup signal can be collected by a preset signal collection device in the vehicle. For example, a microphone is pre-set in the vehicle, and the sound pickup signal of the user when singing is collected by the microphone.

[0082] Optionally, the vehicle-mounted sound pickup signal may be a sound pickup signal directly collected by the signal acquisition device, and then the collected vehicle-mounted sound pickup signal is transmitted to the audio processing device through the signal acquisition device, and the audio signal is preprocessed on the vehicle-mounted sound pickup signal through the audio processing device, and then the preprocessed signal is rendered; optionally, the vehicle-mounted sound pickup signal may also be a processed signal obtained after the signal acquisition device collects the sound pickup signal input by the user and performs the above-mentioned preprocessing on the signal, and the preprocessed vehicle-mounted sound pickup signal is transmitted to the audio processing device for subsequent audio rendering processing. The present application does not specifically limit the specific signal of the vehicle-mounted sound pickup signal.

[0083] Optionally, the sound pickup signal rendering parameters input by the user can also be collected through a preset parameter input device in the car. In actual applications, in order to facilitate the user to understand the meaning of each sound pickup signal rendering parameter, and thus input the sound pickup signal rendering parameter more accurately, the meaning of each parameter can be displayed through a preset display device in the car to assist the user in inputting the parameter. Optionally, the user input can be direct input through an input device (such as a keyboard), or it can be selected by clicking or sliding input. This application does not specifically limit the parameter input method.

[0084] For example, Figure 3 This is a schematic diagram of an interface for inputting sound pickup signal rendering parameters provided by an embodiment of the present application. Figure 3 , the user can directly input or select the pickup signal rendering parameters in the input box displayed in the interface. Optionally, since the vehicle-mounted pickup signal input by the user is the sound of the user singing in the car, the virtual sound source corresponding to the vehicle-mounted pickup signal can be understood as a human voice.

[0085] In an embodiment of the present application, the virtual sound source corresponding to the vehicle-mounted sound pickup signal includes different types of sound sources. Specifically, it can include point sound sources and body sound sources. Optionally, a point sound source can be understood as rendering the singing into a solo at a certain distance; a body sound source can be understood as rendering the singing into a chorus of a nearby choir, or a large range of singing that wraps around the listening position, similar to the sound effect in a confined space such as a bathroom; of course, the sound source type also includes other types of sound sources, such as a line sound source that renders the singing into a chorus of people standing in a row at a certain distance and a surface sound source that renders the singing into a chorus of a choir at a certain distance.

[0086] It also needs to be explained that in actual applications, the singing of birds not far away and the passing cars in the distance can also be regarded as point sound sources; the vibration of guitar strings very close can also be regarded as line sound sources; the vibration of eardrums and large iron sheets nearby can also be regarded as surface sound sources; the roar of car engines nearby can also be regarded as volume sound sources.

[0087] After receiving the sound pickup signal rendering parameters input by the user, the technical solution of the present application also includes: determining the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal based on the sound source shape parameters and the preset shape threshold; if the virtual sound source is determined to be a point sound source, the virtual sound source positions of the auxiliary virtual sound sources are the same, and the sound source position parameters are determined as the virtual sound source position parameters of the virtual sound source; if the virtual sound source is determined to be a volume sound source, the virtual sound source positions of the auxiliary virtual sound sources are different, and the virtual sound source position parameters corresponding to each auxiliary virtual sound source are determined based on the sound source position parameters and the shape parameters.

[0088] In actual applications, different car users will choose different styles of background music to play in the car based on their different preferences, and sing according to the background music played. For example, singing rock-style music in the car, the cheerful style of rock makes the users in the car wrapped in rock sound effects. For another example, singing a singer's concert song in the car, the atmosphere of the concert makes the users in the car have an experience similar to being in a concert. In order to meet the needs of various styles, the technical solution of the present application can pre-set the virtual sound source corresponding to the vehicle-mounted sound pickup signal to include multiple auxiliary virtual sound sources, and then determine the virtual sound source position corresponding to each auxiliary virtual sound source according to the different styles of user preferences, and then perform subsequent rendering processing on the vehicle-mounted sound pickup signal based on the position of each virtual sound source. If the user likes an ethereal playback style, the user can be prompted to input the pickup signal rendering parameters corresponding to the point sound source, and the virtual sound source position parameters of each virtual sound source can be adjusted accordingly, and then subsequent rendering can be performed according to the virtual sound source position parameters, so that the rendered pickup rendering signal can be similar to the ethereal feeling coming from a distant direction when rendered and played; for example, if the user likes an immersive playback style, the user can be prompted to input the pickup signal rendering parameters corresponding to the body sound source, and the position of each virtual sound source can be adjusted according to the pickup signal rendering parameters input by the user, so that the vehicle-mounted sound pickup signal can be similar to the wrapping feeling coming from all directions when rendered and played.

[0089] Specifically, the execution device of the technical solution of the present application may be preset with a sound source type determination module, and the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal input by the vehicle-mounted user is determined based on the sound source type determination module and the sound shape parameter input by the user. Optionally, the sound source type determination module stores a preset shape parameter threshold, obtains the shape parameter threshold, and compares the shape parameter input by the user with the shape parameter threshold, and determines the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal based on the comparison result. Optionally, if the comparison result is that the shape parameter is less than the shape parameter threshold, the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal is determined to be a point sound source. Further, it is determined that the virtual sound source positions of each auxiliary virtual sound source included in the virtual sound source are the same. In other words, the virtual sound source positions of each auxiliary virtual sound source can be approximately regarded as the virtual sound source position of a virtual sound source. On this basis, the sound source position parameter input by the vehicle-mounted user is obtained, and the sound source position parameter is determined as the virtual sound source position parameter of the virtual sound source.

[0090] In practical applications, the sound source position parameters input by the vehicle user can be converted into coordinates based on a preset conversion algorithm to obtain the converted three-dimensional space coordinate parameters as the virtual sound source position parameters required by the virtual sound source in subsequent calculations.

[0091] Optionally, if the comparison result is that the shape parameter is greater than the shape parameter threshold, the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal is determined to be a volume sound source. Further, it is determined that the virtual sound source positions corresponding to the auxiliary virtual sound sources included in the virtual sound source are different. On this basis, the sound source position parameters input by the vehicle-mounted user are obtained, and the sound source position parameters and shape parameters are calculated to determine the virtual sound source position parameters of each auxiliary virtual sound source.

[0092] In practical applications, a position generation algorithm predetermined in the sound source type determination module can be obtained, and the position generation algorithm is used to calculate parameters such as shape parameters and sound source position parameters input by the user to generate virtual sound source position parameters of each virtual sound source. Optionally, the generated virtual sound source position parameters can be three-dimensional space coordinate parameters.

[0093] Exemplarily, when the user selects a uniform shape as the sound source shape when inputting the shape size, and the input sound source shape size is a value α; the virtual sound source position parameter of any auxiliary virtual sound source after the corresponding conversion can be expressed by the following expression; an exemplary expression can be:

[0094]

[0095] in, represents the virtual sound source position parameter of the mth auxiliary virtual sound source after coordinate transformation; represents the virtual sound source position parameter of the mth auxiliary virtual sound source; α represents the sound source shape parameter when the sound source shape parameter input by the user is a uniform shape; Represents the sound source position parameter input by the user.

[0096] Exemplarily, when the user selects an uneven shape when inputting the shape size, and the input sound source shape size includes the sound source width α w , sound source depth α d and the sound source height α h ; The corresponding converted virtual sound source position parameter of any auxiliary virtual sound source can be expressed by the following expression; illustratively, the expression can be:

[0097]

[0098] in, represents the virtual sound source position parameter of the mth auxiliary virtual sound source after coordinate transformation; Represents the virtual sound source position parameter after correction of the mth auxiliary virtual sound source; α w Indicates the width of the sound source when the sound source shape parameter input by the user is an uneven shape; Represents the sound source position parameter input by the user.

[0099] Specifically, the correction process of the virtual sound source position parameters of the auxiliary virtual sound source can be expressed as:

[0100]

[0101] in, Represents the virtual sound source position parameter after correction of the mth auxiliary virtual sound source; α h Indicates the height of the sound source when the sound source shape parameter input by the user is an uneven shape; α d Indicates the depth of the sound source when the sound source shape parameter input by the user is an uneven shape; α w Indicates the width of the sound source when the sound source shape parameter input by the user is an uneven shape; Represents the virtual sound source position parameters of the mth auxiliary virtual sound source.

[0102] It should be noted that, on this basis, when the sound source shape parameter input by the user is a uniform shape,

[0103] Furthermore, based on obtaining the rendering parameters of each sound pickup signal, parameter processing is performed on the sound pickup signal rendering parameters to obtain rendering parameter calculation results corresponding to rendering the vehicle-mounted sound pickup signal to each vehicle-mounted speaker, and then the vehicle-mounted sound pickup signal is accurately rendered based on the rendering parameter calculation results to achieve improved rendering effect.

[0104] Optionally, the rendering parameters of the picked-up signal may be processed to obtain the channel rendering gain by using an amplitude / intensity-based positioning algorithm or a sound field reconstruction-based positioning algorithm; of course, other types of algorithms may also be used, and this application does not limit the selection of the algorithm. Specifically, the amplitude / intensity based positioning algorithm may include a vector-based amplitude / intensity positioning algorithm (e.g., VBAP (Vector-Based Amplitude Panning, vector amplitude translation) / VBIP (Vector-Based Intensity Panning, vector intensity translation)), a multi-directional vector-based amplitude / intensity positioning algorithm (e.g., MDAP (Multiple-Direction Amplitude Panning, multi-directional amplitude translation) / MDIP (Multiple-Direction Intensity Panning, multi-directional intensity translation)), a vector and distance-based amplitude / intensity positioning algorithm (e.g., MD-DBAP (Multiple-Direction–Distance Based Amplitude Panning, multi-directional distance amplitude translation) / MD-DBIP (Multiple-Direction–Distance-Based Intensity Panning, multi-directional intensity amplitude translation)), a multi-directional vector and distance-based amplitude / intensity positioning algorithm (MD-VDBAP (Multiple-Direction–Vector Distance Based Amplitude Panning, multi-directional distance amplitude translation) / MD-DIP (Multiple-Direction–Distance-Based Intensity Panning, multi-directional intensity amplitude translation)), etc. Panning, multi-directional vector distance amplitude translation) / MD-VDBIP (Multiple-Direction–Vector Distance Based Intensity Panning, multi-directional vector intensity amplitude translation) etc.). Specifically, the positioning algorithm based on sound field reconstruction may include: a variety of algorithms based on HOA (High Order Ambisonics, high-order panoramic sound) (IDHOA (Irregular-Layout Decoder for High Order Ambisonics, irregular speaker high-order panoramic sound decoding method), AllRAD (AllRound Ambisonics Decoding, full-round panoramic sound decoding) / AllRAP (All Round Ambisonics Decoding, full-round panoramic sound translation), etc.), a method of synthesizing the sound field using a wave function (for example (WFS (Wave Field Synthesis, wave field synthesis)), Green's function sound field reconstruction, etc.).

[0105] Specifically, the process of obtaining the channel rendering gain through the pickup signal rendering parameter will be described in detail in the subsequent embodiments.

[0106] S220 , rendering the vehicle-mounted sound pickup signal based on the calculation result of the rendering parameter to obtain a sound pickup rendering signal.

[0107] In an embodiment of the present application, based on determining the rendering parameter calculation result based on the sound pickup signal rendering parameter, a preset audio rendering algorithm is obtained, and then the preset audio rendering algorithm is used to render the rendering parameter calculation result and the vehicle-mounted sound pickup signal to obtain a sound pickup rendering signal.

[0108] In practical applications, if the rendering parameter calculation result is a channel rendering gain, an audio rendering algorithm used for rendering based on the gain is obtained, and the channel rendering gain and the vehicle sound pickup signal are processed by the audio rendering algorithm to obtain a processed sound pickup rendering signal. Optionally, the sound pickup rendering signal is a rendering signal of multiple channels.

[0109] Exemplarily, the expression for obtaining the sound pickup rendering signal by rendering based on the gain may include:

[0110] y c [n] = g c x[n]

[0111] Among them, y c [n] represents the rendered output signal of channel c; c is the output channel number of the vehicle speaker; g c represents the channel rendering gain; x[n] represents the vehicle-mounted sound pickup signal.

[0112] In practical applications, if the rendering parameter calculation result is a channel rendering matrix, an audio rendering algorithm used for matrix-based rendering is obtained, and the channel rendering matrix and the vehicle sound pickup signal are processed by the audio rendering algorithm to obtain a processed sound pickup rendering signal. Optionally, the sound pickup rendering signal is a rendering signal of multiple channels.

[0113] Exemplarily, the expression for obtaining the sound pickup rendering signal by performing matrix-based rendering may include:

[0114]

[0115] in, Represents the rendering output signal, C is the number of output channels of the car speaker; A represents the channel rendering matrix, x[n] represents the vehicle-mounted sound pickup signal, 2 indicates the number of input channels for vehicle-mounted sound pickup signals.

[0116] In practical applications, a filtering algorithm can also be used to render the vehicle-mounted sound pickup signal, thereby realizing functions such as reverberation tuning, etc. It should be noted that the filtering algorithm can be used for rendering only when the vehicle-mounted sound pickup signal input by the user is a mono signal.

[0117] Optionally, different filters may be used to render the vehicle sound pickup signal, such as using a FIR (Finite Impulse Response) filter algorithm for rendering, or using an IIR (Infinite Impulse Response) filter algorithm for rendering.

[0118] Exemplary, FIR-based filtering algorithms may include:

[0119]

[0120] Among them, y c [n] represents the output signal of channel c; c is the output channel code of the vehicle speaker; M1 represents the FIR filter order; a c [m] represents the FIR filter coefficient of the cth channel; n represents the current time; m represents the preset time period, x c [nm] represents the input signal at time nm; when m=0, x c [nm] represents the input signal at the current moment; when m>0, x c [nm] represents the input signal at a historical moment.

[0121] Exemplary, IIR-based filtering algorithms may include:

[0122]

[0123] Among them, y c [n] represents the sound pickup rendering signal; c is the output channel code of the car speaker; M1 represents the FIR filter order; a c [m] represents the FIR filter coefficient of the cth channel; M2 represents the IIR filter order; b c [m] represents the FIR filter coefficient of the cth channel; x c [nm] represents the input signal at time nm; when m=0, x c [nm] represents the input signal at the current moment; when m>0, x c [nm] represents the input signal at the historical moment; y c [nm] represents the output signal at a historical moment.

[0124] It should be noted that the audio rendering algorithm for rendering the vehicle-mounted sound pickup signal may also include any rendering algorithm that can be obtained in the prior art, and this application does not specifically limit the rendering algorithm.

[0125] In some possible embodiments, if the vehicle-mounted sound pickup signal transmitted to the audio processing module by the signal acquisition device is the collected initial sound pickup signal, in order to further improve the rendering effect, the vehicle-mounted sound pickup signal needs to be pre-processed before rendering.

[0126] Optionally, the preprocessing process includes but is not limited to: receiving a vehicle-mounted sound pickup signal input by a vehicle-mounted user, and performing signal preprocessing on the vehicle-mounted sound pickup signal to obtain a processed sound pickup signal; obtaining a preset effect generation algorithm, and performing effect processing on the processed sound pickup signal based on the effect generation algorithm to obtain a preprocessed vehicle-mounted sound pickup signal.

[0127] Specifically, signal preprocessing includes noise reduction, reverberation, echo cancellation, and anti-howling preprocessing of the vehicle-mounted sound pickup signal to improve the signal quality and thus improve the subsequent rendering effect.

[0128] In actual applications, users can choose various styles during singing to improve their singing experience. Based on the style selected by the user, the pre-processed sound pickup signal can be processed to make the rendered output audio signal meet the user's style requirements, allowing the user to sing immersively.

[0129] Specifically, an effect generation model can be pre-trained based on audio signals of various styles, and then the processed sound pickup signal and the style information selected by the user are input into the effect generation model to obtain a vehicle sound pickup signal with the above style output by the model, so as to perform subsequent audio rendering processing based on the vehicle sound pickup signal after effect processing.

[0130] S230: Obtain a background rendering signal corresponding to the vehicle-mounted sound pickup signal, perform signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain a target audio signal, and send the target audio signal to each vehicle-mounted speaker.

[0131] In an embodiment of the present application, the background rendering signal may be a rendering signal obtained by rendering a background audio file corresponding to a singing song selected by the user when singing.

[0132] Optionally, in an embodiment of the present application, a specific method for obtaining a background rendering signal corresponding to a vehicle-mounted sound pickup signal, performing signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain a target audio signal may include: obtaining a background audio file corresponding to the vehicle-mounted sound pickup signal, decoding and pre-processing the background audio file to obtain a pre-processed background audio signal; rendering the background audio signal based on a preset audio rendering algorithm to obtain a rendered background rendering signal; and superimposing and mixing the corresponding channel signals of the sound pickup rendering signal and the background rendering signal based on a preset signal mixing algorithm to obtain a target audio signal.

[0133] In order to improve the user's singing experience in the car, after selecting the song to sing, the user can further select whether the background music played contains the original song. Furthermore, the background audio file obtained according to the user's selection can be the original audio file, or a background audio file containing only the accompaniment data after the original audio file is separated from the sound accompaniment.

[0134] On the basis of obtaining the background audio file, the background audio file is decoded to obtain a modulated background audio signal, and then the background audio signal is rendered. Optionally, a preset audio rendering algorithm may be obtained, and then the background audio signal is rendered based on the audio rendering signal to obtain a corresponding background rendering signal. It should be noted that the audio rendering algorithm used in the above-mentioned rendering of the background audio may be any rendering algorithm in the prior art, which may be the same as or different from the audio rendering algorithm corresponding to the above-mentioned pickup signal, and the present application does not specifically limit the selection of the audio algorithm.

[0135] Furthermore, based on respectively rendering the sound pickup rendering signal and the background rendering signal, signal mixing processing is performed on the sound pickup rendering signal and the background rendering signal to obtain a mixed target audio signal, and the target audio signal is transmitted to each speaker for audio playback.

[0136] In a possible embodiment, a 12-channel sound pickup rendering signal can be obtained by performing multi-channel rendering processing on a mono / dual-channel vehicle-mounted sound pickup signal; and a 12-channel background rendering signal can be obtained by performing multi-channel rendering processing on a 6-channel background audio signal; and then the 12-channel sound pickup rendering signal and the 12-channel background rendering signal are mixed to obtain a target audio signal.

[0137] It should be noted that when rendering the background audio signal, the background audio signal containing stereo sound can be up-mixed into a multi-channel signal inside the car; for example, 7.1.4, 5.1.4, etc., and the background audio signal containing high-order panoramic sound can be down-mixed into a multi-channel signal inside the car; for example, 7.1.4, 5.1.4, etc.

[0138] It should also be noted that when mixing the sound pickup rendering signal and the background rendering signal based on the corresponding channels, the sound pickup rendering of the first channel and the background rendering signal of the first channel can be mixed to obtain the target audio signal of the first channel; and then the target audio signal of 12 channels can be obtained based on the corresponding channel mixing method.

[0139] In the above technical scheme, before rendering the vehicle-mounted sound pickup signal input by the user, the rendering parameter calculation result corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal is determined in advance based on the sound pickup signal rendering parameter input by the user, and the vehicle-mounted sound pickup signal is rendered based on the rendering parameter calculation result to obtain the sound pickup rendering signal, so as to achieve accurate rendering of the sound pickup signal; at the same time, the background audio signal corresponding to the vehicle-mounted sound pickup signal is rendered to obtain the corresponding background rendering signal, and the rendered sound pickup rendering signal and the background rendering signal are mixed and transmitted to the speaker for playback, thereby achieving accurate rendering of the audio signal in the scenario where the speakers in the car are irregularly arranged, thereby optimizing the overall rendering effect in the car.

[0140] Figure 4 This is a flow chart of another in-vehicle audio rendering method provided by an embodiment of the present application. This embodiment can be understood as a specific introduction embodiment of the steps mentioned in the method described in the above embodiment. Figure 4 , the method may specifically include:

[0141] S310: Acquire an in-vehicle sound pickup signal and a rendering parameter of the sound pickup signal input by an in-vehicle user, and determine a rendering parameter calculation result corresponding to rendering the in-vehicle sound pickup signal to each in-vehicle speaker based on the rendering parameter of the sound pickup signal.

[0142] Specifically, the understanding and examples of the technical means, technical effects, and technical terms in step S310 may refer to the explanation of step S210 in the above embodiment.

[0143] On the basis of the above implementation, in this embodiment, the judging step in step S310 may specifically include:

[0144] S311 . Determine a rendering mode corresponding to rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source based on the listening position parameter and the virtual sound source position parameter.

[0145] In an embodiment of the present application, in order to achieve accurate rendering of the vehicle-mounted sound pickup signal in the case of an irregular speaker layout in the vehicle, it is necessary to pre-acquire the listening position parameters of the user's listening position in the vehicle and the virtual sound source position parameters of the virtual sound source corresponding to the vehicle-mounted sound pickup signal when rendering, and determine the rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source based on the listening position parameters and the virtual sound source position parameters, and then select the corresponding gain calculation method based on the rendering mode, and determine the rendering parameter calculation result corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method.

[0146] Among them, the rendering mode can be understood as the different rendering situations corresponding to the virtual sound source being located at different positions in the sound field when rendering the vehicle-mounted sound pickup signal to the virtual sound source, when the sound field is determined based on the speaker position parameters of the speaker and the listening position parameters of the listening position. In practical applications, the rendering mode may include near-field rendering or far-field rendering. Specifically, near-field rendering can be understood as the rendering situation corresponding to the virtual sound source being inside the sound field; far-field rendering can be understood as the rendering situation corresponding to the virtual sound source being outside the sound field.

[0147] Optionally, the speaker position parameters can be obtained by reading data from a database storing vehicle parameters; the listening position parameters and the virtual sound source position parameters can be obtained by reading the sound pickup signal rendering parameters input by the user, and then the above-mentioned position parameters are calculated based on a preset calculation expression, and then the corresponding rendering mode is determined based on the calculation result when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source.

[0148] It needs to be explained that in actual applications, in order to accurately render the audio signal in the car, multiple crossover speakers and multiple channel speakers will be pre-arranged in the vehicle. Figure 5 This is a schematic diagram of the layout of the vehicle speakers provided in the embodiment of the present application. Figure 5 , the figure includes multiple vehicle speakers arranged around the main driver. Specifically, it includes a left front top channel speaker arranged on the top of the left door, a center channel speaker arranged in front of the center console, and other channel speakers, as well as a left front channel high-frequency speaker arranged on the vehicle A-pillar and a left front channel mid-low-frequency speaker arranged on the left door.

[0149] Specifically, for different types of speakers, the speaker position parameters are determined in different ways. Optionally, for channel speakers, the channel spatial position of each speaker can be determined as the speaker position parameter of each channel speaker; for any group of crossover speakers of different frequencies, the channel spatial positions corresponding to the crossover speakers of different frequencies can be weighted averaged and other calculations can be performed, and the calculation results obtained by the processing can be used as the speaker position parameters of the group of speakers of different frequencies.

[0150] Exemplarily, for the channel speakers, the left front top channel spatial position of the left front top channel speaker is its corresponding speaker position parameter; the center channel spatial position of the center channel speaker is its corresponding speaker position parameter; for the crossover speakers, the left front channel high-frequency speaker and the left front channel mid-low-frequency speaker are a group of crossover speakers of different frequencies, and the left front channel spatial position corresponding to the left front channel high-frequency speaker and the left front mid-low-frequency spatial position corresponding to the left front channel mid-low-frequency speaker are weighted averaged to obtain the calculated channel spatial position, and the channel spatial position is determined as the speaker position parameter corresponding to the left front channel high-frequency speaker and the left front channel mid-low-frequency speaker.

[0151] It is also necessary to explain that if the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal is different when rendering, the virtual sound source positions of the auxiliary virtual sound sources included in the virtual sound source corresponding to the vehicle-mounted sound pickup signal obtained subsequently are also different. Optionally, if the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a point sound source, the virtual sound source containing multiple auxiliary virtual sound sources can be regarded as an auxiliary virtual sound source, and then the rendering mode corresponding to the rendering of the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source is determined based on the virtual sound source position parameters and the listening position parameters of the auxiliary virtual sound source. Optionally, if the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a volume sound source, then for any auxiliary virtual sound source, the rendering mode corresponding to the rendering of the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source can be determined based on the listening position parameters and the virtual sound source position parameters of the current auxiliary virtual sound source.

[0152] Optionally, the method for determining the rendering mode in the technical solution of the present application may specifically include: obtaining the sound field threshold for rendering the vehicle-mounted speaker signal, and comparing the virtual sound source position parameter and the sound field threshold; if the virtual sound source position parameter is within the sound field threshold range, determining the rendering mode as near-field rendering; if the virtual sound source position parameter is outside the sound field threshold range, determining the rendering mode as far-field rendering.

[0153] Among them, the sound field threshold can be understood as the radius value of the sound field sphere constructed with the listening position in the car as the origin. Among them, the sound field threshold can be set in advance together with other vehicle parameters, and of course it can also be determined based on the speaker position of each speaker in the car. Specifically, the speaker position parameters of each speaker in the car are obtained, and the distances between each speaker and the listening position are determined based on the speaker position parameters and the listening position parameters, and the distances of each position are sorted to obtain the farthest position distance between the speaker and the listening position, and then the farthest position distance is determined as the sound field threshold; of course, other methods can also be used to determine the sound field threshold, and this application does not make specific restrictions on this. It should be noted that when calculating the farthest position distance between the speaker and the listening position, the calculation can be based on various types of speakers in the car. In practical applications, various types of speakers include but are not limited to horizontal speakers and sky speakers.

[0154] Specifically, when the sound field threshold is determined, the virtual sound source position parameter of the virtual sound source corresponding to the vehicle-mounted sound pickup signal is compared with the sound field threshold, and the rendering mode corresponding to the vehicle-mounted sound pickup signal is determined based on the result of the comparison. Optionally, if the virtual sound source position parameter is less than the sound field threshold, it means that the virtual sound source position parameter is within the sound field threshold range, and the rendering mode is determined to be near-field rendering; if the virtual sound source position parameter is greater than the sound field threshold, it means that the virtual sound source position parameter is outside the sound field threshold range, and the rendering mode is determined to be far-field rendering; optionally, if the virtual sound source position parameter is equal to the sound field threshold, the rendering mode of the vehicle-mounted sound pickup signal can be specifically determined according to the actual situation, that is, it can be determined as near-field rendering or far-field rendering, and the method of determining the rendering mode in this case is not specifically limited.

[0155] For example, Figure 6 Schematic diagram of the cross section of the sound field sphere constructed in the embodiment of the present application. Figure 6 In the figure, point 1 is the listening position; points L, C, R, Lss, Rss, Lrs, and Rrs are the locations of the horizontal speakers; Ltf, Rtf, Ltb, and Rtb are the locations of the sky speakers; the circle constructed with point 1 as the origin and the distance between point 1 and point Rrs as the radius is the cross-section of the sound field sphere; further, the inside of the sound field sphere is the near field, and the outside of the sound field sphere is the far field. The corresponding rendering mode for sound source 1 is near field rendering, and the rendering mode for sound source 2 is far field rendering.

[0156] It should be noted that, in the above-mentioned process of determining the rendering mode of the vehicle-mounted sound pickup signal during rendering, if it is predetermined that the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a point sound source, then the auxiliary virtual sound sources included in the virtual sound source can be regarded as a virtual sound source for subsequent determination of the rendering mode; if it is predetermined that the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a volume sound source, then the rendering mode corresponding to each auxiliary virtual sound source in the virtual sound source can be calculated separately, and then the rendering parameter calculation results corresponding to each auxiliary virtual sound source can be calculated separately, and then the rendering parameter calculation results corresponding to the vehicle-mounted sound pickup signal can be determined based on multiple rendering parameter calculation results.

[0157] S312: Obtain a gain calculation method corresponding to the rendering mode, and determine, based on the gain calculation method, a channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source.

[0158] In the embodiment of the present application, the rendering parameter calculation result may be a channel rendering gain. Accordingly, the gain calculation methods corresponding to various rendering modes may be obtained, the virtual sound source position parameters of the virtual sound source corresponding to the vehicle-mounted sound pickup signal may be determined, and the virtual sound source position parameters may be brought into the gain calculation method as calculation parameters, thereby obtaining the calculated channel rendering gain.

[0159] It should be noted that if the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal is different during rendering, the virtual sound source position parameters corresponding to the vehicle-mounted sound pickup signal obtained subsequently are also different. Optionally, if the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a point sound source, the virtual sound source containing multiple auxiliary virtual sound sources can be regarded as an auxiliary virtual sound source, and then the gain calculation method is determined based on the virtual sound source position parameters of the auxiliary virtual sound source, and the channel rendering gain is calculated.

[0160] Optionally, when the virtual sound source of the vehicle-mounted sound pickup signal is a point sound source and the rendering mode is near-field rendering, a gain calculation method corresponding to the near-field rendering is obtained, and based on the gain calculation method, a channel rendering gain corresponding to the vehicle-mounted speaker channel system when the vehicle-mounted sound pickup signal is rendered to the virtual sound source position of the virtual sound source is determined; a specific channel rendering gain calculation method may include: determining a first distance between each vehicle-mounted speaker and the virtual sound source position corresponding to the vehicle-mounted sound pickup signal based on a virtual sound source position parameter and a speaker position parameter of each vehicle-mounted speaker, and determining a normalized parameter for rendering the vehicle-mounted sound pickup signal based on each first distance; determining a channel rendering gain corresponding to the vehicle-mounted speaker channel system when the vehicle-mounted sound pickup signal is rendered to the virtual sound source position of the virtual sound source based on the normalized parameter and each first distance.

[0161] Specifically, since the total sound of all speakers in the near field can maintain energy conservation, on this basis, when calculating the channel rendering gain, the position distance between each speaker and the virtual sound source position can be determined first, and the normalization parameter can be determined based on the position, and then the channel rendering gain can be obtained based on the normalized parameter and the above-mentioned position distance.

[0162] Exemplarily, taking any speaker as an example, the position parameters of the speaker and the position parameters of the virtual sound source position can be obtained, and then the position distance between the speaker and the virtual sound source position can be determined based on the two position parameters, and then the position distances between each speaker and the virtual sound source position can be determined based on the above method.

[0163] On this basis, a preset normalized calculation expression can be obtained, and then the normalized parameters can be determined based on the normalized calculation expression and each position distance. Exemplarily, the normalized calculation expression may include:

[0164]

[0165] Where, k represents the normalization parameter; L represents the number of speakers in the car; d sl Represents the first distance, that is, the position distance between the loudspeaker and the virtual sound source.

[0166] On this basis, a channel rendering gain calculation expression of the speaker in the near field rendering mode is obtained, and the first distance and the normalized parameter are substituted into the above expression for calculation to obtain the channel rendering gain. Exemplarily, the channel rendering gain calculation expression of the speaker in the near field rendering mode may include:

[0167]

[0168] Among them, g l represents the channel rendering gain of the speaker in the near-field rendering mode; k represents the normalization parameter; d sl Represents the first distance, that is, the position distance between the loudspeaker and the virtual sound source.

[0169] Optionally, when the virtual sound source of the vehicle-mounted sound pickup signal is a point sound source and the rendering mode is far-field rendering, the gain calculation method corresponding to the far-field rendering is obtained, and based on the gain calculation method, the channel rendering gain corresponding to the vehicle-mounted speaker channel system when the vehicle-mounted sound pickup signal is rendered to the virtual sound source position of the virtual sound source is determined; the specific channel rendering gain calculation method may include: determining the second distance between the virtual sound source position corresponding to the vehicle-mounted sound pickup signal and the listening position of the vehicle-mounted user based on the virtual sound source position parameter and the listening position parameter; obtaining the attenuation parameter of the vehicle-mounted sound pickup signal rendering, and determining the normalization parameter of the vehicle-mounted sound pickup signal rendering based on the attenuation parameter and each first distance, and determining the channel rendering gain corresponding to the vehicle-mounted speaker channel system when the vehicle-mounted sound pickup signal is rendered to the virtual sound source position of the virtual sound source based on the normalization parameter and each second distance.

[0170] Specifically, since the total sound of all speakers in the far field will become smaller as the distance between the virtual sound source and the listening position increases, on this basis, when calculating the channel rendering gain, the position distance between the listening position and the virtual sound source position can be first determined, and the normalization parameter can be determined based on the position distance and the acquired attenuation parameter, and then the channel rendering gain can be obtained based on the normalization parameter and the above position distance.

[0171] Optionally, the attenuation parameter can be obtained by calling parameters in a database of pre-stored vehicle parameters; of course, it can also be determined based on speaker position parameters, virtual sound source position parameters and listening position parameters. Specifically, the speaker position parameters and listening position parameters are calculated using the above implementation to determine the farthest position distance between the listening position and the speakers, and the position distance between the listening position and the virtual sound source position is determined based on the position parameters of the listening position and the position parameters of the virtual sound source position; further, the attenuation parameter is determined based on the ratio of the farthest position distance and the position distance between the listening position and the virtual sound source position. Exemplarily, the expression of the attenuation parameter may include:

[0172]

[0173] Where p represents the attenuation parameter; d max Indicates the farthest distance between the listening position and the speakers; d rs Indicates the distance between the listening position and the virtual sound source position.

[0174] It should be noted that in the far field, p<1, and p decreases as the distance between the virtual sound source and the listening position increases, so the total energy of the speaker is p 2 <1, that is, getting smaller and smaller.

[0175] On this basis, a preset normalized calculation expression can be obtained, and then the normalized parameters can be determined based on the normalized calculation expression and each position distance. Exemplarily, the normalized calculation expression may include:

[0176]

[0177] Where, k represents the normalization parameter; p represents the attenuation parameter; L represents the number of speakers in the car; d sl Represents the first distance, that is, the position distance between the loudspeaker and the virtual sound source.

[0178] On this basis, a channel rendering gain calculation expression of the speaker in the near field rendering mode is obtained, and the first distance and the normalized parameter are substituted into the above expression for calculation to obtain the channel rendering gain. Exemplarily, the channel rendering gain calculation expression of the speaker in the near field rendering mode may include:

[0179]

[0180] Among them, g l represents the channel rendering gain of the speaker in far-field rendering mode; k represents the normalization parameter; d sl Represents the first distance, that is, the position distance between the loudspeaker and the virtual sound source.

[0181] Optionally, if the virtual sound source corresponding to the vehicle-mounted sound pickup signal is a volume sound source, the virtual sound source position parameters of multiple auxiliary virtual sound sources may be included in the virtual sound source respectively, and then the gain calculation method is determined based on the virtual sound source position parameters of each auxiliary virtual sound source, and the channel rendering gain is calculated. Specifically, when the virtual sound source of the vehicle-mounted sound pickup signal is a volume sound source, the method for determining the channel rendering gain may include: obtaining the gain calculation method corresponding to the rendering mode, determining the sound source rendering gain corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the gain calculation method; and determining the channel rendering gain corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the sound source rendering gain of each auxiliary virtual sound source.

[0182] On the basis of the above implementation mode, the above processing method is used to determine the sound source rendering gain corresponding to each virtual auxiliary sound source based on the virtual sound source position parameters of each virtual auxiliary sound source, and the rendering speaker gain of each sound source is processed to obtain the channel rendering gain corresponding to rendering the vehicle-mounted sound pickup signal to each vehicle-mounted speaker.

[0183] Exemplarily, the channel rendering gain may be obtained based on the following expression:

[0184]

[0185] in, Indicates the channel rendering gain corresponding to the vehicle-mounted sound pickup signal rendered to each vehicle-mounted speaker; Indicates the channel rendering gain corresponding to the mth virtual sound source; g L,m represents the channel rendering gain of the Lth loudspeaker corresponding to the mth virtual sound source; M represents the number of sound sources of the virtual sound source; Representation vector The second-order norm of can be calculated as:

[0186] S320: Render the vehicle-mounted sound pickup signal based on the calculation result of the rendering parameter to obtain a sound pickup rendering signal.

[0187] Specifically, the understanding and examples of the technical means, technical effects, and technical terms in step S320 may refer to the explanation of step S220 in the above embodiment.

[0188] S330: Obtain a background rendering signal corresponding to the vehicle-mounted sound pickup signal, perform signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain a target audio signal, and send the target audio signal to each vehicle-mounted speaker.

[0189] Specifically, for the understanding and examples of the technical means, technical effects, and technical terms in step S330, reference may be made to the explanation of step S230 in the above embodiment.

[0190] In the above scheme, before rendering the vehicle-mounted sound pickup signal input by the user, the rendering parameter calculation result corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal is determined in advance based on the sound pickup signal rendering parameters input by the user, wherein the rendering parameter calculation result may be a channel rendering gain; optionally, the sound source type of the virtual sound source of the vehicle-mounted sound pickup signal and the rendering mode when the virtual sound source renders the vehicle-mounted sound pickup signal may be determined based on the sound pickup signal rendering parameters, and then the corresponding gain calculation method is determined based on the sound source type and the rendering mode, and the channel rendering gain is obtained based on the gain calculation method, and then the vehicle-mounted sound pickup signal is rendered based on the channel rendering gain to obtain a sound pickup rendering signal, so as to achieve accurate rendering of the sound pickup signal; at the same time, the background audio signal corresponding to the vehicle-mounted sound pickup signal is rendered to obtain the corresponding background rendering signal, and the rendered sound pickup rendering signal and the background rendering signal are mixed and transmitted to the speaker for playback, thereby achieving accurate rendering of the audio signal in a scenario where the speakers in the vehicle are irregularly arranged, thereby achieving optimization of the overall rendering effect in the vehicle.

[0191] Figure 7 This is a schematic diagram of the structure of a vehicle audio rendering device provided by an embodiment of the present application. Figure 7 The device comprises: a rendering parameter calculation result determination module 410, a sound pickup rendering signal acquisition module 420 and a target audio signal acquisition module 430; wherein,

[0192] The rendering parameter calculation result determination module 410 is used to obtain the vehicle-mounted sound pickup signal and the sound pickup signal rendering parameter input by the vehicle-mounted user, and determine the rendering parameter calculation result corresponding to the vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal based on the sound pickup signal rendering parameter;

[0193] A rendering parameter calculation result module 420 is used to render the vehicle-mounted sound pickup signal based on the rendering parameter calculation result to obtain a sound pickup rendering signal;

[0194] The target audio signal acquisition module 430 is used to obtain the background rendering signal corresponding to the vehicle-mounted sound pickup signal, perform signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain the target audio signal, and send the target audio signal to each vehicle-mounted speaker.

[0195] Optionally, the sound pickup signal rendering parameters include a listening position parameter of the vehicle user and a sound source parameter of a virtual sound source corresponding to the vehicle sound pickup signal;

[0196] The sound source parameters include sound source position parameters and sound source shape parameters; the sound source position parameters include sound source distance and sound source angle; the sound source shape parameters include uniform shape parameters or non-uniform shape parameters;

[0197] Uniform shape parameters include the uniform size of the sound source; non-uniform shape parameters include the height, width and depth of the sound source.

[0198] Optionally, the virtual sound source corresponding to the vehicle-mounted sound pickup signal includes multiple auxiliary virtual sound sources;

[0199] Accordingly, the device further comprises:

[0200] A virtual sound source position determination module is used to determine the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal based on the sound source shape parameter and a preset shape threshold after obtaining the sound pickup signal rendering parameter input by the vehicle-mounted user; the sound source type includes a point sound source and a volume sound source;

[0201] A first virtual sound source position parameter determination module, configured to, if the virtual sound source is determined to be a point sound source, have the virtual sound source positions of the auxiliary virtual sound sources be the same, and determine the sound source position parameters as the virtual sound source position parameters of the virtual sound source;

[0202] The second virtual sound source position parameter determination module is used to determine the virtual sound source position parameters corresponding to each auxiliary virtual sound source based on the sound source position parameter and the shape parameter if the virtual sound source is determined to be a volume sound source, then the virtual sound source positions of the auxiliary virtual sound sources are different.

[0203] Optionally, the rendering parameter calculation result includes channel rendering gain, if the virtual sound source is a point sound source;

[0204] Accordingly, the rendering parameter calculation result determination module 410 includes:

[0205] A rendering mode determination submodule, used to determine a corresponding rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source based on the listening position parameter and the virtual sound source position parameter;

[0206] The channel rendering gain determination submodule is used to obtain the gain calculation method corresponding to the rendering mode, and determine the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method.

[0207] Optionally, the rendering mode includes near-field rendering or far-field rendering;

[0208] Correspondingly, the rendering mode determination submodule includes:

[0209] A parameter comparison unit, used to obtain the sound field threshold of the vehicle speaker signal rendering, and compare the virtual sound source position parameter with the sound field threshold;

[0210] a near-field rendering mode determination unit, configured to determine that the rendering mode is near-field rendering if the virtual sound source position parameter is within a sound field threshold range;

[0211] The far-field rendering mode determination unit is used to determine the rendering mode as far-field rendering if the virtual sound source position parameter is outside the sound field threshold range.

[0212] Optional, if the rendering mode is near-field rendering;

[0213] Correspondingly, the channel rendering gain determination submodule includes:

[0214] a normalization parameter determination unit, configured to determine a first distance between each vehicle-mounted speaker and a virtual sound source position corresponding to the vehicle-mounted sound pickup signal based on the virtual sound source position parameter and the speaker position parameter of each vehicle-mounted speaker, and determine a normalization parameter for rendering the vehicle-mounted sound pickup signal based on each first distance;

[0215] The first channel rendering gain determining unit is used to determine, based on the normalized parameter and each first distance, a channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source.

[0216] Optional, if the rendering mode is far field rendering;

[0217] Correspondingly, the channel rendering gain determination submodule includes:

[0218] A normalization parameter determination unit, used to obtain an attenuation parameter of the vehicle-mounted sound pickup signal rendering, and determine a normalization parameter of the vehicle-mounted sound pickup signal rendering based on the attenuation parameter and each second distance;

[0219] The second channel rendering gain determining unit is used to determine, based on the normalized parameter and each second distance, a channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source.

[0220] The rendering parameter calculation results include channel rendering gain, if the virtual sound source is determined to be a volume sound source;

[0221] Accordingly, the rendering parameter calculation result determination module 410 includes:

[0222] A rendering mode determination submodule, for determining, for any auxiliary virtual sound source, a rendering mode corresponding to rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the listening position parameter and the virtual sound source position parameter of the current auxiliary virtual sound source;

[0223] A sound source rendering gain determination submodule, used to obtain a gain calculation method corresponding to a rendering mode, and determine a sound source rendering gain corresponding to a vehicle speaker channel system when rendering a vehicle sound pickup signal to a virtual sound source position of a current auxiliary virtual sound source based on the gain calculation method;

[0224] The channel rendering gain determination submodule is used to determine the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the sound source rendering gain of each auxiliary virtual sound source.

[0225] Optionally, the target audio signal obtaining module 430 includes:

[0226] The background audio signal acquisition submodule is used to obtain the background audio file corresponding to the vehicle-mounted sound pickup signal, and decode and pre-process the background audio file to obtain the pre-processed background audio signal;

[0227] A background rendering signal obtaining submodule is used to render the background audio signal based on a preset audio rendering algorithm to obtain a rendered background rendering signal;

[0228] The target audio signal acquisition submodule is used to perform superposition and mixing processing on the corresponding channel signals of the picked-up rendering signal and the background rendering signal based on a preset signal mixing algorithm to obtain the target audio signal.

[0229] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device of this embodiment may include:

[0230] at least one processor 510; and

[0231] A memory 520 communicatively coupled to at least one processor;

[0232] The memory 520 stores instructions that can be executed by at least one processor 510, and the instructions are executed by at least one processor 510 to enable the server to execute a method as described in any of the above embodiments.

[0233] Optionally, the memory 520 may be independent or integrated with the processor 510 .

[0234] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0235] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method of any of the aforementioned embodiments is implemented.

[0236] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method of any of the aforementioned embodiments when executed by a processor.

[0237] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0238] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0239] It should be understood that the above-mentioned processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0240] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0241] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in a server or a main control device.

[0242] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present application. Fig. 9 , device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0243] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0244] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0245] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0246] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0247] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0248] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0249] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0250] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0251] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0252] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0253] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned split-screen processing method of the electronic device.

[0254] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0255] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A vehicle audio rendering method, characterized in that: The method comprises: Acquire an in-vehicle sound pickup signal and a rendering parameter of the sound pickup signal input by an in-vehicle user, and determine a rendering parameter calculation result corresponding to an in-vehicle speaker channel system when rendering the in-vehicle sound pickup signal based on the rendering parameter of the sound pickup signal; Rendering the vehicle-mounted sound pickup signal based on the rendering parameter calculation result to obtain a sound pickup rendering signal; A background rendering signal corresponding to the vehicle-mounted sound pickup signal is obtained, the sound pickup rendering signal and the background rendering signal are mixed to obtain a target audio signal, and the target audio signal is sent to each of the vehicle-mounted speakers.

2. The method according to claim 1, characterized in that The sound pickup signal rendering parameters include the listening position parameters of the vehicle user and the sound source parameters of the virtual sound source corresponding to the vehicle sound pickup signal; The sound source parameters include sound source position parameters and sound source shape parameters; the sound source position parameters include sound source distance and sound source angle; the sound source shape parameters include uniform shape parameters or non-uniform shape parameters; The uniform shape parameters include the uniform size of the sound source; the non-uniform shape parameters include the height, width and depth of the sound source.

3. The method according to claim 2, characterized in that The virtual sound source corresponding to the vehicle-mounted sound pickup signal includes a plurality of auxiliary virtual sound sources; Accordingly, after obtaining the rendering parameters of the sound pickup signal input by the vehicle user, the method further includes: Determine the sound source type of the virtual sound source corresponding to the vehicle-mounted sound pickup signal based on the sound source shape parameter and a preset shape threshold; the sound source type includes a point sound source and a volume sound source; If it is determined that the virtual sound source is a point sound source, the virtual sound source positions of the auxiliary virtual sound sources are the same, and the sound source position parameters are determined as the virtual sound source position parameters of the virtual sound source; If the virtual sound source is determined to be a volume sound source, the virtual sound source positions of the auxiliary virtual sound sources are different, and virtual sound source position parameters corresponding to the auxiliary virtual sound sources are determined based on the sound source position parameters and the shape parameters.

4. The method according to claim 3, characterized in that The rendering parameter calculation result includes channel rendering gain, if the virtual sound source is a point sound source; Correspondingly, the calculation result of the rendering parameter corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal based on the sound pickup signal rendering parameter includes: Determine, based on the listening position parameter and the virtual sound source position parameter, a corresponding rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source; A gain calculation method corresponding to the rendering mode is obtained, and based on the gain calculation method, a channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined.

5. The method according to claim 4, characterized in that The rendering mode includes near-field rendering or far-field rendering; Correspondingly, the determining, based on the listening position parameter and the virtual sound source position parameter, of a corresponding rendering mode when rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the virtual sound source includes: Acquiring a sound field threshold rendered by the vehicle-mounted speaker signal, and performing a comparison process on the virtual sound source position parameter and the sound field threshold; If the virtual sound source position parameter is within the sound field threshold range, determining that the rendering mode is near-field rendering; If the virtual sound source position parameter is outside the sound field threshold range, the rendering mode is determined to be far-field rendering.

6. The method according to claim 4, characterized in that If the rendering mode is near-field rendering; Accordingly, determining the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method includes: Determine a first distance between each vehicle-mounted speaker and the virtual sound source position corresponding to the vehicle-mounted sound pickup signal based on the virtual sound source position parameter and the speaker position parameter of each vehicle-mounted speaker, and determine a normalized parameter for rendering the vehicle-mounted sound pickup signal based on each of the first distances; A channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined based on the normalized parameter and each of the first distances.

7. The method according to claim 6, characterized in that If the rendering mode is far field rendering; Accordingly, determining the channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the virtual sound source based on the gain calculation method includes: Acquire an attenuation parameter of the vehicle-mounted sound pickup signal rendering, and determine a normalization parameter of the vehicle-mounted sound pickup signal rendering based on the attenuation parameter and each of the first distances; A channel rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the virtual sound source is determined based on the normalized parameter and each of the first distances.

8. The method according to claim 3, characterized in that The rendering parameter calculation result includes a channel rendering gain, if it is determined that the virtual sound source is a volume sound source; Correspondingly, the calculation result of the rendering parameter corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal based on the sound pickup signal rendering parameter includes: For any auxiliary virtual sound source, determining a rendering mode corresponding to rendering the vehicle-mounted sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source based on the listening position parameter and the virtual sound source position parameter of the current auxiliary virtual sound source; Acquire a gain calculation method corresponding to the rendering mode, and determine, based on the gain calculation method, a sound source rendering gain corresponding to a vehicle speaker channel system when rendering the vehicle sound pickup signal to a virtual sound source position of the current auxiliary virtual sound source; Based on the sound source rendering gain of each of the auxiliary virtual sound sources, a channel rendering gain corresponding to the vehicle speaker channel system when rendering the vehicle sound pickup signal to the virtual sound source position of the current auxiliary virtual sound source is determined.

9. The method according to claim 1, characterized in that: The step of obtaining a background rendering signal corresponding to the vehicle-mounted sound pickup signal, and performing signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain a target audio signal includes: Acquire a background audio file corresponding to the vehicle-mounted sound pickup signal, and perform decoding preprocessing on the background audio file to obtain a preprocessed background audio signal; Based on a preset audio rendering algorithm, the background audio signal is rendered to obtain a rendered background rendering signal; Based on a preset signal mixing algorithm, the corresponding channel signals of the sound pickup rendering signal and the background rendering signal are superimposed and mixed to obtain a target audio signal.

10. A vehicle-mounted audio rendering device, characterized in that: The device comprises: A rendering parameter calculation result determination module, used to obtain a vehicle-mounted sound pickup signal and a sound pickup signal rendering parameter input by a vehicle-mounted user, and determine a rendering parameter calculation result corresponding to a vehicle-mounted speaker channel system when rendering the vehicle-mounted sound pickup signal based on the sound pickup signal rendering parameter; A sound pickup rendering signal obtaining module, used for rendering the vehicle-mounted sound pickup signal based on the rendering parameter calculation result to obtain a sound pickup rendering signal; The target audio signal acquisition module is used to obtain the background rendering signal corresponding to the vehicle-mounted sound pickup signal, perform signal mixing processing on the sound pickup rendering signal and the background rendering signal to obtain the target audio signal, and send the target audio signal to each of the vehicle-mounted speakers.

11. An electronic device, characterized in that: include: A processor and a memory communicatively connected to the processor; The memory stores computer-executable instructions; When executing the computer-executable instructions, the processor is used to implement the in-vehicle audio rendering method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the in-vehicle audio rendering method according to any one of claims 1 to 10.

Images